From nobody@FreeBSD.org  Thu Feb 14 07:55:46 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
	by hub.freebsd.org (Postfix) with ESMTP id C339C585
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Feb 2013 07:55:46 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id B58CDB9E
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Feb 2013 07:55:46 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.5/8.14.5) with ESMTP id r1E7tkxs093159
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 14 Feb 2013 07:55:46 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.5/8.14.5/Submit) id r1E7tkRU093158;
	Thu, 14 Feb 2013 07:55:46 GMT
	(envelope-from nobody)
Message-Id: <201302140755.r1E7tkRU093158@red.freebsd.org>
Date: Thu, 14 Feb 2013 07:55:46 GMT
From: Akinori MUSHA <knu@FreeBSD.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: cp(1) fails to overwrite a symlink pointing to a directory
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         176136
>Category:       bin
>Synopsis:       [patch] cp(1) fails to overwrite a symlink pointing to a directory
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    markj
>State:          analyzed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Feb 14 08:00:00 UTC 2013
>Closed-Date:    
>Last-Modified:  Mon Feb 18 17:50:00 UTC 2013
>Originator:     Akinori MUSHA
>Release:        9.1-STABLE
>Organization:
>Environment:
FreeBSD 9.1-STABLE/amd64
>Description:
Our implementation of cp(1) cannot overwrite a symlink pointing to an existing directory.
This is because it always calls stat(2) on a destination file path even if -R (with -P by default) is given, where lstat(2) should be used instead.

Dragonfly BSD, OS X and OpenBSD all share the same problem with us.
NetBSD cp(1) has once had this problem but fixed it in 2006 (bin/cp.c rev.1.42).
GNU cp does not have this problem.
Solaris/OpenIndiana's cp(1) (/usr/xpg4/bin/cp) seems to have this problem.

As far as I read, SUSv4 is not very clear as to how cp(1) should do about the situation, but there should be no reason a symlink cannot be overwritten.

>How-To-Repeat:
% ln -s . foo
% mkdir bar
% ln -s .. bar/foo

Now we have <foo> and <bar/foo>.
Let's try copying <foo> to <bar/foo>.

% cp -RP foo bar/
cp: cannot overwrite directory bar/foo with non-directory foo

It failed!

% ls -l bar
total 1
lrwxr-xr-x  1 knu  knu  2 Feb 14 16:18 foo -> ..

It seems cp(1) decided not to copy the source file just because the destination symlink was resolved to a directory, but it is certainly not what you would expect.
>Fix:
Index: cp.c
===================================================================
--- cp.c	(revision 246770)
+++ cp.c	(working copy)
@@ -262,7 +262,7 @@ copy(char *argv[], enum op type, int fts
 	struct stat to_stat;
 	FTS *ftsp;
 	FTSENT *curr;
-	int base = 0, dne, badcp, rval;
+	int base = 0, dne, badcp, rval, sval;
 	size_t nlen;
 	char *p, *target_mid;
 	mode_t mask, mode;
@@ -383,8 +383,18 @@ copy(char *argv[], enum op type, int fts
 			continue;
 		}
 
+		/*
+                 * lstat(2) should be used if neither -H or -L is
+                 * given, or we will fail to overwrite an existing
+                 * symlink pointing to a directory.
+                 */
+		if (fts_options & (FTS_LOGICAL | FTS_COMFOLLOW))
+			sval = stat(to.p_path, &to_stat);
+                else
+			sval = lstat(to.p_path, &to_stat);
+
 		/* Not an error but need to remember it happened */
-		if (stat(to.p_path, &to_stat) == -1)
+		if (sval == -1)
 			dne = 1;
 		else {
 			if (to_stat.st_dev == curr->fts_statp->st_dev &&


>Release-Note:
>Audit-Trail:

From: "Akinori MUSHA" <knu@FreeBSD.org>
To: FreeBSD-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/176136: cp(1) fails to overwrite a symlink pointing to a directory
Date: Thu, 14 Feb 2013 17:19:41 +0900

 Seems I messed up the patch and blew up tabs, so I put it on Gist:
 
 https://gist.github.com/knu/4951299
 
 -- 
 Akinori MUSHA / http://akinori.org/
State-Changed-From-To: open->analyzed 
State-Changed-By: markj 
State-Changed-When: Mon Feb 18 05:04:24 UTC 2013 
State-Changed-Why:  
I'll take it. 


Responsible-Changed-From-To: freebsd-bugs->markj 
Responsible-Changed-By: markj 
Responsible-Changed-When: Mon Feb 18 05:04:24 UTC 2013 
Responsible-Changed-Why:  
I'll take it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=176136 

From: Mark Johnston <markj@freebsd.org>
To: bug-followup@FreeBSD.org, knu@FreeBSD.org
Cc:  
Subject: Re: bin/176136: [patch] cp(1) fails to overwrite a symlink pointing
 to a directory
Date: Mon, 18 Feb 2013 00:02:26 -0500

 Hi,
 
 Could you give the patch at [1] a try? I think your patch is correct but
 doesn't preserve cp's behaviour of allowing users to create files by
 copying through a dangling symlink, as Jilles pointed out in bin/174489.
 
 My patch just handles that as a special case but otherwise just chooses
 between stat and lstat based on the -P flag, which is what NetBSD does.
 If it works for you as well I'll commit it in a few days.
 
 Thanks,
 -Mark
 
 [1] http://people.freebsd.org/~markj/patches/cp/002_stat_vs_lstat.diff

From: "Akinori MUSHA" <knu@FreeBSD.org>
To: Mark Johnston <markj@freebsd.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/176136: [patch] cp(1) fails to overwrite a symlink pointing to a directory
Date: Mon, 18 Feb 2013 14:27:41 +0900

 At Mon, 18 Feb 2013 00:02:26 -0500,
 Mark Johnston wrote:
 > Could you give the patch at [1] a try? I think your patch is correct but
 
 Thanks, the worked for me.
 
 > doesn't preserve cp's behaviour of allowing users to create files by
 > copying through a dangling symlink, as Jilles pointed out in bin/174489.
 
 Ah, I should have been well aware of the behavior but my patch lacked
 further testing.
 
 > My patch just handles that as a special case but otherwise just chooses
 > between stat and lstat based on the -P flag, which is what NetBSD does.
 > If it works for you as well I'll commit it in a few days.
 
 Yes, please.
 
 By the way, I think we need some regression test suite for utilities
 like cp(1), or do we have one already?
 
 -- 
 Akinori MUSHA / http://akinori.org/

From: Mark Johnston <markj@freebsd.org>
To: Akinori MUSHA <knu@FreeBSD.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/176136: [patch] cp(1) fails to overwrite a symlink pointing
 to a directory
Date: Mon, 18 Feb 2013 12:23:06 -0500

 On Mon, Feb 18, 2013 at 02:27:41PM +0900, Akinori MUSHA wrote:
 > At Mon, 18 Feb 2013 00:02:26 -0500,
 > Mark Johnston wrote:
 > > Could you give the patch at [1] a try? I think your patch is correct but
 > 
 > Thanks, the worked for me.
 > 
 > > doesn't preserve cp's behaviour of allowing users to create files by
 > > copying through a dangling symlink, as Jilles pointed out in bin/174489.
 > 
 > Ah, I should have been well aware of the behavior but my patch lacked
 > further testing.
 
 It's a FreeBSD-specific behaviour as far as I know.
 
 > 
 > > My patch just handles that as a special case but otherwise just chooses
 > > between stat and lstat based on the -P flag, which is what NetBSD does.
 > > If it works for you as well I'll commit it in a few days.
 > 
 > Yes, please.
 > 
 > By the way, I think we need some regression test suite for utilities
 > like cp(1), or do we have one already?
 
 We don't have one at the moment. I've been working on a little test
 suite which I'll eventually add to tools/regression/.

From: "Akinori MUSHA" <knu@FreeBSD.org>
To: Mark Johnston <markj@freebsd.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/176136: [patch] cp(1) fails to overwrite a symlink pointing to a directory
Date: Tue, 19 Feb 2013 02:47:02 +0900

 At Mon, 18 Feb 2013 12:23:06 -0500,
 Mark Johnston wrote:
 > On Mon, Feb 18, 2013 at 02:27:41PM +0900, Akinori MUSHA wrote:
 > > At Mon, 18 Feb 2013 00:02:26 -0500,
 > > Mark Johnston wrote:
 > > > Could you give the patch at [1] a try? I think your patch is correct but
 > > 
 > > Thanks, the worked for me.
 > > 
 > > > doesn't preserve cp's behaviour of allowing users to create files by
 > > > copying through a dangling symlink, as Jilles pointed out in bin/174489.
 > > 
 > > Ah, I should have been well aware of the behavior but my patch lacked
 > > further testing.
 > 
 > It's a FreeBSD-specific behaviour as far as I know.
 
 That's a surprise to me, since I knew "echo hello > dangling_link"
 would create a new file on most platforms and thought so does cp(1)
 everywhere.  I've confirmed that GNU cp rejects copying on a dangling
 link while FreeBSD and NetBSD cp accept it.
 
 > > > My patch just handles that as a special case but otherwise just chooses
 > > > between stat and lstat based on the -P flag, which is what NetBSD does.
 > > > If it works for you as well I'll commit it in a few days.
 > > 
 > > Yes, please.
 > > 
 > > By the way, I think we need some regression test suite for utilities
 > > like cp(1), or do we have one already?
 > 
 > We don't have one at the moment. I've been working on a little test
 > suite which I'll eventually add to tools/regression/.
 
 The way our cp(1) and mv(1) recognizes and handles a trailing slash is
 also a implementation specific feature, which in my opinion is quite
 nice.
 
 Having a regression test will greatly help preserve such features
 whenever one tries to refactor the code.
 
 -- 
 Akinori MUSHA / http://akinori.org/
>Unformatted:
