From nobody@FreeBSD.org  Sun Dec 16 23:16:45 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id E613E578
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 16 Dec 2012 23:16:45 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id CB1108FC14
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 16 Dec 2012 23:16:45 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.5/8.14.5) with ESMTP id qBGNGjss001130
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 16 Dec 2012 23:16:45 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.5/8.14.5/Submit) id qBGNGjRB001129;
	Sun, 16 Dec 2012 23:16:45 GMT
	(envelope-from nobody)
Message-Id: <201212162316.qBGNGjRB001129@red.freebsd.org>
Date: Sun, 16 Dec 2012 23:16:45 GMT
From: "Micha&#322; G&#243;rny" <mgorny@gentoo.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: 'cp -a -n' fails when 'over-writing' a broken symlink
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         174489
>Category:       bin
>Synopsis:       cp(1): 'cp -a -n' fails when 'over-writing' a broken symlink
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    markj
>State:          analyzed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Dec 16 23:20:01 UTC 2012
>Closed-Date:    
>Last-Modified:  Sun Jan 27 06:00:00 UTC 2013
>Originator:     Micha&#322; G&#243;rny
>Release:        8.2-RELEASE-p2
>Organization:
>Environment:
FreeBSD sand 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #0: Sun Aug 21 21:47:45 UTC 2011     root@onepiece:/secure/usr/obj/secure/usr/src/sys/ONE-PIECE  amd64

>Description:
If 'cp -a -n' is used to copy a tree of files containing a 'broken' symlink for the second time, the call fails with the following error:

    cp: symlink: python-exec: File exists

(with 'python-exec' being the non-existent target of the broken symlink, not the symlink name)

I would assume that the no-clobber behavior enforced by '-n' should actually ignore the symlink, broken or not.

The bug was originally reported on Gentoo Bugzilla because of an eclass not working with BSD version of 'cp' [0]. A detailed analysis of the problem has been submitted there by Mike Gilbert, citing:

> Inside of copy() in cp.c [1], the dne variable is set to 1 based on a call
> to stat(2). stat(2) attempts to follow symlinks, and returns -1 when called
> on the easy_install -> python-exec symlink because python-exec doesn't exist
> in ${D}.
> 
> !dne is then passed to copy_link in utils.c [2]. copy_link is supposed to
> remove the destination if it exists (via unlink), but the bad value in dne
> causes this to be skipped.
> 
> Since the destination file is not removed, the call to symlink then fails,
> producing the "cp: symlink: python-exec: File exists" message.
> 
> [1] http://svnweb.freebsd.org/base/head/bin/cp/cp.c?view=markup
> [2] http://svnweb.freebsd.org/base/head/bin/cp/utils.c?view=markup

[0]:https://bugs.gentoo.org/show_bug.cgi?id=447370
>How-To-Repeat:
mkdir 1 2
ln -s python-exec 1/foo
cp -a -n 1/* 2/
cp -a -n 1/* 2/
>Fix:
Citing Mike Gilbert:

> I think this could be fixed by changing the stat(2) to lstat(2) in cp.c.
> copy_link should probably be changed to obey the "-n" flag (nflag) as well.


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->markj 
Responsible-Changed-By: markj 
Responsible-Changed-When: Fri Dec 21 04:43:17 UTC 2012 
Responsible-Changed-Why:  
I'll take it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=174489 

From: Mark Johnston <markj@freebsd.org>
To: bug-followup@freebsd.org, mgorny@gentoo.org
Cc:  
Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 &#39;over-writing&#39; a broken symlink
Date: Thu, 20 Dec 2012 23:56:44 -0500

 As noted, there are two bugs here:
 1. The "exists" flag in copy_link() is set based on whether the target
    can be stat(2)'ed. But if -R is specified and the destination is a
    symlink, the "exists" flag ends up referring to the path that the
    target symlink is pointing to. The result is that the target symlink
    isn't unlink(2)ed before symlink(2) is called in copy_link(), so
    cp(1) errors out.
 2. copy_link() doesn't respect -n. So in the following example, baz
    should be left unchanged by cp(1), but we're overwriting it:
 
   $ touch foo1 foo2
   $ ln -s foo1 bar
   $ ln -s foo2 baz
   $ cp -an bar baz
   $ ls -la bar bar
   lrwxr-xr-x  1 mark  mark  4 Dec 20 23:49 bar -> foo1
   lrwxr-xr-x  1 mark  mark  4 Dec 20 23:49 baz -> foo1
   $
 
 The attached patch fixes both of these issues. I'd also like to add
 some regression tests for cp(1) (we have a bunch for mv(1)), but in the
 meantime, can you test the patch?
 
 Thanks,
 -Mark

From: Mark Johnston <markj@freebsd.org>
To: bug-followup@freebsd.org, mgorny@gentoo.org
Cc:  
Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 &#39;over-writing&#39; a broken symlink
Date: Thu, 20 Dec 2012 23:59:52 -0500

 --e89a8fb204f4ce805104d155b78b
 Content-Type: text/plain; charset=ISO-8859-1
 
 On Thu, Dec 20, 2012 at 11:56 PM, Mark Johnston <markj@freebsd.org> wrote:
 ...
 > The attached patch fixes both of these issues. I'd also like to add
 > some regression tests for cp(1) (we have a bunch for mv(1)), but in the
 > meantime, can you test the patch?
 
 And I of course forgot to actually attach it. Sorry about that.
 
 --e89a8fb204f4ce805104d155b78b
 Content-Type: application/octet-stream; name="cp_dash_n_and_symlinks.patch"
 Content-Disposition: attachment; filename="cp_dash_n_and_symlinks.patch"
 Content-Transfer-Encoding: base64
 X-Attachment-Id: f_hayuo29u0
 
 ZGlmZiAtLWdpdCBhL2Jpbi9jcC9jcC5jIGIvYmluL2NwL2NwLmMKaW5kZXggODZkYmIzYy4uMjQy
 OTZkZSAxMDA2NDQKLS0tIGEvYmluL2NwL2NwLmMKKysrIGIvYmluL2NwL2NwLmMKQEAgLTM4Niw3
 ICszODYsOCBAQCBjb3B5KGNoYXIgKmFyZ3ZbXSwgZW51bSBvcCB0eXBlLCBpbnQgZnRzX29wdGlv
 bnMpCiAJCX0KIAogCQkvKiBOb3QgYW4gZXJyb3IgYnV0IG5lZWQgdG8gcmVtZW1iZXIgaXQgaGFw
 cGVuZWQgKi8KLQkJaWYgKHN0YXQodG8ucF9wYXRoLCAmdG9fc3RhdCkgPT0gLTEpCisJCWlmICgo
 IVJmbGFnICYmIHN0YXQodG8ucF9wYXRoLCAmdG9fc3RhdCkgPT0gLTEpIHx8CisJCSAgICAoUmZs
 YWcgJiYgbHN0YXQodG8ucF9wYXRoLCAmdG9fc3RhdCkgPT0gLTEpKQogCQkJZG5lID0gMTsKIAkJ
 ZWxzZSB7CiAJCQlpZiAodG9fc3RhdC5zdF9kZXYgPT0gY3Vyci0+ZnRzX3N0YXRwLT5zdF9kZXYg
 JiYKQEAgLTQ4MSw3ICs0ODIsNyBAQCBjb3B5KGNoYXIgKmFyZ3ZbXSwgZW51bSBvcCB0eXBlLCBp
 bnQgZnRzX29wdGlvbnMpCiAJCQkJYmFkY3AgPSBydmFsID0gMTsKIAkJCWJyZWFrOwogCQl9Ci0J
 CWlmICh2ZmxhZyAmJiAhYmFkY3ApCisJCWlmICh2ZmxhZyAmJiAhYmFkY3AgJiYgIW5mbGFnKQog
 CQkJKHZvaWQpcHJpbnRmKCIlcyAtPiAlc1xuIiwgY3Vyci0+ZnRzX3BhdGgsIHRvLnBfcGF0aCk7
 CiAJfQogCWlmIChlcnJubykKZGlmZiAtLWdpdCBhL2Jpbi9jcC91dGlscy5jIGIvYmluL2NwL3V0
 aWxzLmMKaW5kZXggZDcyOWJkNS4uOGNlZGViYSAxMDA2NDQKLS0tIGEvYmluL2NwL3V0aWxzLmMK
 KysrIGIvYmluL2NwL3V0aWxzLmMKQEAgLTI2Niw2ICsyNjYsMTEgQEAgY29weV9saW5rKGNvbnN0
 IEZUU0VOVCAqcCwgaW50IGV4aXN0cykKIAlpbnQgbGVuOwogCWNoYXIgbGxpbmtbUEFUSF9NQVhd
 OwogCisJaWYgKGV4aXN0cyAmJiBuZmxhZykgeworCQlpZiAodmZsYWcpCisJCQlwcmludGYoIiVz
 IG5vdCBvdmVyd3JpdHRlblxuIiwgdG8ucF9wYXRoKTsKKwkJcmV0dXJuICgwKTsKKwl9CiAJaWYg
 KChsZW4gPSByZWFkbGluayhwLT5mdHNfcGF0aCwgbGxpbmssIHNpemVvZihsbGluaykgLSAxKSkg
 PT0gLTEpIHsKIAkJd2FybigicmVhZGxpbms6ICVzIiwgcC0+ZnRzX3BhdGgpOwogCQlyZXR1cm4g
 KDEpOwo=
 --e89a8fb204f4ce805104d155b78b--
State-Changed-From-To: open->feedback 
State-Changed-By: markj 
State-Changed-When: Fri Dec 21 19:38:21 UTC 2012 
State-Changed-Why:  
Ask for submitter approval. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=174489 

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, mgorny@gentoo.org
Cc:  
Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 &#39;over-writing&#39; a broken symlink
Date: Mon, 24 Dec 2012 21:26:15 +0100

 PR bin/174489:
 > [overwriting a symlink with cp -an]
 
 The patch appears to fix overwriting a symlink with a symlink, but
 breaks creating a symlink's destination by copying a regular file onto
 it.
 
 Example:
 :>testf1; ln -fs testf2 testl1; cp -R testf1 testl1;
 ls -l testf2; rm testf1 testf2 testl1
 
 These commands should all succeed.
 
 I am not entirely sure but it may be appropriate to set
 int follow_link = (fts_options & FTS_LOGICAL) ||
     ((fts_options & FTS_COMFOLLOW) && curr->fts_level == 0);
 and call
 (follow_link ? stat : lstat).
 
 Note that follow_link will always be true if -R was not given.
 
 The FreeBSD kernel also allows creating a symlink's destination as a
 directory by passing a pathname ending in a slash (if there is no slash,
 POSIX requires mkdir() to fail) but cp does not expose this
 functionality.
 
 -- 
 Jilles Tjoelker

From: Mark Johnston <markj@freebsd.org>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@freebsd.org
Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 &#39;over-writing&#39; a broken symlink
Date: Tue, 25 Dec 2012 12:37:04 -0500

 On Mon, Dec 24, 2012 at 08:30:01PM +0000, Jilles Tjoelker wrote:
 > The following reply was made to PR bin/174489; it has been noted by GNATS.
 > 
 > From: Jilles Tjoelker <jilles@stack.nl>
 > To: bug-followup@FreeBSD.org, mgorny@gentoo.org
 > Cc:  
 > Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 >  &#39;over-writing&#39; a broken symlink
 > Date: Mon, 24 Dec 2012 21:26:15 +0100
 > 
 >  PR bin/174489:
 >  > [overwriting a symlink with cp -an]
 >  
 >  The patch appears to fix overwriting a symlink with a symlink, but
 >  breaks creating a symlink's destination by copying a regular file onto
 >  it.
 >  
 >  Example:
 >  :>testf1; ln -fs testf2 testl1; cp -R testf1 testl1;
 >  ls -l testf2; rm testf1 testf2 testl1
 >  
 >  These commands should all succeed.
 
 Ok. It's probably worth noting somewhere that this goes against what
 POSIX says, since open("<dangling symlink>", O_WRONLY | O_TRUNC) will
 fail, and cp(1) should then bail by section 3.a.iii. We need to add
 O_CREAT for the above example to work properly.
 
 >  
 >  I am not entirely sure but it may be appropriate to set
 >  int follow_link = (fts_options & FTS_LOGICAL) ||
 >      ((fts_options & FTS_COMFOLLOW) && curr->fts_level == 0);
 >  and call
 >  (follow_link ? stat : lstat).
 >  
 >  Note that follow_link will always be true if -R was not given.
 
 That logic doesn't quite work: if -R is given (and -L and -H are not),
 then fts_options and thus follow_link will be 0, so lstat(2) will return
 0 and we'll thus try to open <dst> without O_CREAT, leading to an error.
 
 I think we just need to add a special check for a broken symlink as the
 destination path. I'll try to come up with a patch shortly.
 
 >  
 >  The FreeBSD kernel also allows creating a symlink's destination as a
 >  directory by passing a pathname ending in a slash (if there is no slash,
 >  POSIX requires mkdir() to fail) but cp does not expose this
 >  functionality.
 >  
 >  -- 
 >  Jilles Tjoelker

From: =?UTF-8?B?TWljaGHFgiBHw7Nybnk=?= <mgorny@gentoo.org>
To: Mark Johnston <markj@freebsd.org>
Cc: bug-followup@freebsd.org
Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 &#39;over-writing&#39; a broken symlink
Date: Thu, 10 Jan 2013 15:50:18 +0100

 --Sig_/Mdex_kGZ=W.6i_Ij.GKxgJO
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable
 
 On Thu, 20 Dec 2012 23:56:44 -0500
 Mark Johnston <markj@freebsd.org> wrote:
 
 > As noted, there are two bugs here:
 > 1. The "exists" flag in copy_link() is set based on whether the target
 >    can be stat(2)'ed. But if -R is specified and the destination is a
 >    symlink, the "exists" flag ends up referring to the path that the
 >    target symlink is pointing to. The result is that the target symlink
 >    isn't unlink(2)ed before symlink(2) is called in copy_link(), so
 >    cp(1) errors out.
 > 2. copy_link() doesn't respect -n. So in the following example, baz
 >    should be left unchanged by cp(1), but we're overwriting it:
 >=20
 >   $ touch foo1 foo2
 >   $ ln -s foo1 bar
 >   $ ln -s foo2 baz
 >   $ cp -an bar baz
 >   $ ls -la bar bar
 >   lrwxr-xr-x  1 mark  mark  4 Dec 20 23:49 bar -> foo1
 >   lrwxr-xr-x  1 mark  mark  4 Dec 20 23:49 baz -> foo1
 >   $
 >=20
 > The attached patch fixes both of these issues. I'd also like to add
 > some regression tests for cp(1) (we have a bunch for mv(1)), but in the
 > meantime, can you test the patch?
 
 I've finally got the OP to reply on the patch [1]. Although it
 seems to fix the original issue, it seems to break '-v' option:
 
 $ cp -a -l -v -n 1/* 3/
 $ cp -a -l -v -n 1/* 3/
 3/easy_install not overwritten
 
 (note that first 'cp' invocation didn't print copied files)
 
 $ cp -a -l -v 1/* 3/
 1/easy_install -> 3/easy_install
 
 [1]:https://bugs.gentoo.org/show_bug.cgi?id=3D447370
 
 --=20
 Best regards,
 Micha=C5=82 G=C3=B3rny
 
 --Sig_/Mdex_kGZ=W.6i_Ij.GKxgJO
 Content-Type: application/pgp-signature; name=signature.asc
 Content-Disposition: attachment; filename=signature.asc
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.19 (GNU/Linux)
 
 iJwEAQEIAAYFAlDu1S0ACgkQfXuS5UK5QB30uwP/Y5uHrFPUmizPaJpDj4ZVDwhJ
 7jLFWSYOwugiXMct59jA4MxcSG8N5RzfMLVBYYyKNR+3VG8nX1+g4Fp13EcRCGX/
 1qRWWjKFC7lgmnxCtdoCnwOq/f2sUjtqkCTf24iyS1Yw9egXNyDKlBd7q7EAdJqN
 fwAAy5MhNmRKHErhhmo=
 =2WoL
 -----END PGP SIGNATURE-----
 
 --Sig_/Mdex_kGZ=W.6i_Ij.GKxgJO--
State-Changed-From-To: feedback->analyzed 
State-Changed-By: markj 
State-Changed-When: Wed Jan 23 18:17:19 UTC 2013 
State-Changed-Why:  
The reported issues are fixed with the attached patch. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=174489 

From: Mark Johnston <markj@freebsd.org>
To: =?utf-8?B?TWljaGHFgiBHw7Nybnk=?= <mgorny@gentoo.org>
Cc: bug-followup@freebsd.org
Subject: Re: bin/174489: cp(1): &#39;cp -a -n&#39; fails when
 &#39;over-writing&#39; a broken symlink
Date: Wed, 23 Jan 2013 11:15:53 -0800

 On Thu, Jan 10, 2013 at 03:50:18PM +0100, Michał Górny wrote:
 > On Thu, 20 Dec 2012 23:56:44 -0500
 > Mark Johnston <markj@freebsd.org> wrote:
 > 
 > > As noted, there are two bugs here:
 > > 1. The "exists" flag in copy_link() is set based on whether the target
 > >    can be stat(2)'ed. But if -R is specified and the destination is a
 > >    symlink, the "exists" flag ends up referring to the path that the
 > >    target symlink is pointing to. The result is that the target symlink
 > >    isn't unlink(2)ed before symlink(2) is called in copy_link(), so
 > >    cp(1) errors out.
 > > 2. copy_link() doesn't respect -n. So in the following example, baz
 > >    should be left unchanged by cp(1), but we're overwriting it:
 > > 
 > >   $ touch foo1 foo2
 > >   $ ln -s foo1 bar
 > >   $ ln -s foo2 baz
 > >   $ cp -an bar baz
 > >   $ ls -la bar bar
 > >   lrwxr-xr-x  1 mark  mark  4 Dec 20 23:49 bar -> foo1
 > >   lrwxr-xr-x  1 mark  mark  4 Dec 20 23:49 baz -> foo1
 > >   $
 > > 
 > > The attached patch fixes both of these issues. I'd also like to add
 > > some regression tests for cp(1) (we have a bunch for mv(1)), but in the
 > > meantime, can you test the patch?
 > 
 > I've finally got the OP to reply on the patch [1]. Although it
 > seems to fix the original issue, it seems to break '-v' option:
 > 
 > $ cp -a -l -v -n 1/* 3/
 > $ cp -a -l -v -n 1/* 3/
 > 3/easy_install not overwritten
 > 
 > (note that first 'cp' invocation didn't print copied files)
 > 
 > $ cp -a -l -v 1/* 3/
 > 1/easy_install -> 3/easy_install
 
 Ok, thanks for the feedback. My current patches for this problem are
 here: http://people.freebsd.org/~markj/patches/cp/
 Specifically, the first two patches resolve the original issues reported
 in this PR (the last is for another bug). I haven't looked into the
 problem with the -v option yet, but I'll do that now.
 
 Thanks,
 -Mark

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/174489: commit references a PR
Date: Sun, 27 Jan 2013 05:59:36 +0000 (UTC)

 Author: markj
 Date: Sun Jan 27 05:59:28 2013
 New Revision: 245960
 URL: http://svnweb.freebsd.org/changeset/base/245960
 
 Log:
   Return with an error from copy_link(), copy_fifo() and copy_special() if
   the -n option is specified and the destination file exists.
   
   PR:		bin/174489
   Approved by:	rstone (co-mentor)
   MFC after:	2 weeks
 
 Modified:
   head/bin/cp/utils.c
 
 Modified: head/bin/cp/utils.c
 ==============================================================================
 --- head/bin/cp/utils.c	Sun Jan 27 05:45:55 2013	(r245959)
 +++ head/bin/cp/utils.c	Sun Jan 27 05:59:28 2013	(r245960)
 @@ -266,6 +266,11 @@ copy_link(const FTSENT *p, int exists)
  	int len;
  	char llink[PATH_MAX];
  
 +	if (exists && nflag) {
 +		if (vflag)
 +			printf("%s not overwritten\n", to.p_path);
 +		return (1);
 +	}
  	if ((len = readlink(p->fts_path, llink, sizeof(llink) - 1)) == -1) {
  		warn("readlink: %s", p->fts_path);
  		return (1);
 @@ -285,6 +290,12 @@ copy_link(const FTSENT *p, int exists)
  int
  copy_fifo(struct stat *from_stat, int exists)
  {
 +
 +	if (exists && nflag) {
 +		if (vflag)
 +			printf("%s not overwritten\n", to.p_path);
 +		return (1);
 +	}
  	if (exists && unlink(to.p_path)) {
  		warn("unlink: %s", to.p_path);
  		return (1);
 @@ -299,6 +310,12 @@ copy_fifo(struct stat *from_stat, int ex
  int
  copy_special(struct stat *from_stat, int exists)
  {
 +
 +	if (exists && nflag) {
 +		if (vflag)
 +			printf("%s not overwritten\n", to.p_path);
 +		return (1);
 +	}
  	if (exists && unlink(to.p_path)) {
  		warn("unlink: %s", to.p_path);
  		return (1);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
