From tamaru@gin.myn.rcast.u-tokyo.ac.jp  Tue Dec 23 00:16:55 2003
Return-Path: <tamaru@gin.myn.rcast.u-tokyo.ac.jp>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 3265E16A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 23 Dec 2003 00:16:55 -0800 (PST)
Received: from gin.myn.rcast.u-tokyo.ac.jp (cognac.myn.rcast.u-tokyo.ac.jp [157.82.72.158])
	by mx1.FreeBSD.org (Postfix) with ESMTP id B13C843D2D
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 23 Dec 2003 00:16:52 -0800 (PST)
	(envelope-from tamaru@gin.myn.rcast.u-tokyo.ac.jp)
Received: from gin.myn.rcast.u-tokyo.ac.jp (localhost [127.0.0.1])
	by gin.myn.rcast.u-tokyo.ac.jp (8.12.10/8.12.10) with ESMTP id hBN8Gn9I084810
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 23 Dec 2003 17:16:49 +0900 (JST)
	(envelope-from tamaru@gin.myn.rcast.u-tokyo.ac.jp)
Received: (from tamaru@localhost)
	by gin.myn.rcast.u-tokyo.ac.jp (8.12.10/8.12.10/Submit) id hBN8GmR5084809;
	Tue, 23 Dec 2003 17:16:48 +0900 (JST)
	(envelope-from tamaru)
Message-Id: <200312230816.hBN8GmR5084809@gin.myn.rcast.u-tokyo.ac.jp>
Date: Tue, 23 Dec 2003 17:16:48 +0900 (JST)
From: Hiroharu Tamaru <tamaru@myn.rcast.u-tokyo.ac.jp>
Reply-To: Hiroharu Tamaru <tamaru@myn.rcast.u-tokyo.ac.jp>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: mtree tumbles on files begining with hashmark(#)
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         60524
>Category:       bin
>Synopsis:       mtree tumbles on files begining with hashmark(#)
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    simokawa
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Dec 23 00:20:10 PST 2003
>Closed-Date:    Wed Jan 12 21:52:17 GMT 2005
>Last-Modified:  Wed Jan 12 21:52:17 GMT 2005
>Originator:     Hiroharu Tamaru
>Release:        FreeBSD 4.9-STABLE i386
>Organization:
The University of Tokyo
>Environment:
System: FreeBSD gin 4.9-STABLE FreeBSD 4.9-STABLE #0: Fri Nov 28 15:13:39 JST 2003 MYN-P6 i386


	
>Description:
	
mtree fails to handle filenames that begin with hashmark(#).
It writes the mtree file fine in -c mode, but skips the line
as a comment line in check mode and thus gives false alarm.
>How-To-Repeat:
	
echo a > #a
echo b > a
echo c > a#
echo d > '\a'
echo e > 'a\'
echo f > '\#a'
echo g > b
ls -l
total 7
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 #a
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 \#a
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 \a
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 a
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 a#
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 a\
-rw-r--r--  1 tamaru  wheel  2 Dec 23 16:55 b
mtree -c -K md5digest > /tmp/test.mtree
mtree < /tmp/test.mtree
. changed
        MD5 expected 60b725f10c9c85c70d97880dfe8191b3 found 1335b892002f8dcf803\
e474f33efdb11
#a extra

cat /tmp/test.mtree
(snip)
# .
/set type=file uid=0 gid=0 mode=0644 nlink=1 flags=none
.               type=dir uid=10001 mode=0755 nlink=2 size=512 \
                time=1072166150.0
    #a          uid=10001 size=2 time=1072166144.0 \
                md5digest=60b725f10c9c85c70d97880dfe8191b3
    \\#a        uid=10001 size=2 time=1072166144.0 \
                md5digest=9a8ad92c50cae39aa2c5604fd0ab6d8c
    \\a         uid=10001 size=2 time=1072166144.0 \
                md5digest=e29311f6f1bf1af907f9ef9f44b8328b
    a           uid=10001 size=2 time=1072166144.0 \
                md5digest=3b5d5c3712955042212316173ccf37be
    a#          uid=10001 size=2 time=1072166144.0 \
                md5digest=2cd6ee2c70b0bde53fbe6cac3c8b8bb1
    a\\         uid=10001 size=2 time=1072166144.0 \
                md5digest=9ffbf43126e33be52cd2bf7e01d627f9
    b           uid=10001 size=2 time=1072166144.0 \
                md5digest=f5302386464f953ed581edac03556e55
..

>Fix:

	
Since backslashes are escaped correctly, I suggest that a leading hashmark
be escaped as '\#'.
Adding a backslash before # by hand in a mtree file does not solve the
problem, so I assume that both writeing and reading routines should be
updated.
Curretly, as I see it, a file named '\#a' is written in mtree file as
'\\#a', but an mtree file that reads '\#a' is also interpreted as filename
'\#a', which may or may not be a real compatibility issue.

I haven't dug into the source to come up with a patch, sorry.
>Release-Note:
>Audit-Trail:

From: Hiroharu Tamaru <tamaru@myn.rcast.u-tokyo.ac.jp>
To: freebsd-gnats-submit@FreeBSD.org
Cc: phk@FreeBSD.org
Subject: Re: bin/60524: mtree tumbles on files begining with hashmark(#)
Date: Thu, 25 Dec 2003 01:38:52 +0900

 To follow up my own PR...
 
 I found that the problem was fixed in
 revision 1.32 of usr.sbin/mtree/create.c and
 revision 1.13 of lib/libc/gen/vis.c
 for -current and 5.2-RELEASE.
 I definitely would like to see an MFC.
 
 TIA
 
 http://www.freebsd.org/cgi/query-pr.cgi?pr=60524
State-Changed-From-To: open->patched 
State-Changed-By: anholt 
State-Changed-When: Tue Jan 6 21:21:37 PST 2004 
State-Changed-Why:  
Fixed in -current but not MFCed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60524 
State-Changed-From-To: patched->closed 
State-Changed-By: kris 
State-Changed-When: Sat Apr 24 00:49:05 PDT 2004 
State-Changed-Why:  
MFC will not be performed since the change breaks compatibility with older mtree files. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60524 

From: Hiroharu Tamaru <tamaru@myn.rcast.u-tokyo.ac.jp>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/60524: mtree tumbles on files begining with hashmark(#)
Date: Mon, 26 Apr 2004 02:22:15 +0900

 Forgot to CC it to gnats...
 Date: Mon, 26 Apr 2004 00:18:48 +0900
 From: Hiroharu Tamaru <tamaru@myn.rcast.u-tokyo.ac.jp>
 To: Kris Kennaway <kris@FreeBSD.org>
 Cc: freebsd-bugs@FreeBSD.org
 Subject: Re: bin/60524: mtree tumbles on files begining with hashmark(#)
 
 Thank you for looking into this Kris.
 
 At Sat, 24 Apr 2004 00:49:25 -0700 (PDT),
 Kris Kennaway wrote:
 > 
 > Synopsis: mtree tumbles on files begining with hashmark(#)
 > 
 > State-Changed-From-To: patched->closed
 > State-Changed-By: kris
 > State-Changed-When: Sat Apr 24 00:49:05 PDT 2004
 > State-Changed-Why: 
 > MFC will not be performed since the change breaks compatibility with older mtree files.
 > 
 > http://www.freebsd.org/cgi/query-pr.cgi?pr=60524
 
 Since the actual fix committed to -current is a more robust
 one than that I proposed, I believe the changes are fully
 backward compatible.  It uses octal escapes (such as \043
 for #) that had been supported by mtree all the time.  That
 is to say:
 
 An old mtree file that doesn't have any special chars is
 readable by new mtree program. 
 
 An old mtree file that have special chars like # is broken
 anyways and is unreadable by both old and new mtree program
 
 A new mtree file that doesn't have any special chars is
 readable by old mtree program.
 
 A new mtree file that have special chars is readable by old
 mtree program.
 
 That said, are there other compatibility issues that I
 couldn't come up with?  Please point me to the archives if
 there was an discussion.
 
 Thank you.
 
 
 I'll attach a patch against -stable as of today for your
 reference.  It MFC's revision 1.32 of create.c, revisions
 1.11, 1.12 and 1.13 of vis.c, and revisions 1.19 thru 1.23
 of vis.3.
 
 You could also consider MFC'ing 1.9 and 1.10 for vis.c.
 
 --- usr.sbin/mtree/create.c.orig	Sun Apr 25 23:36:22 2004
 +++ usr.sbin/mtree/create.c	Sun Apr 25 23:37:52 2004
 @@ -159,7 +159,7 @@
  	escaped_name = calloc(1, p->fts_namelen * 4  +  1);
  	if (escaped_name == NULL)
  		errx(1, "statf(): calloc() failed");
 -	strvis(escaped_name, p->fts_name, VIS_WHITE | VIS_OCTAL);
 +	strvis(escaped_name, p->fts_name, VIS_WHITE | VIS_OCTAL | VIS_GLOB);
  
  	if (iflag || S_ISDIR(p->fts_statp->st_mode))
  		offset = printf("%*s%s", indent, "", escaped_name);
 --- lib/libc/gen/vis.c.orig	Sun Apr 25 23:36:06 2004
 +++ lib/libc/gen/vis.c	Sun Apr 25 23:38:03 2004
 @@ -71,7 +71,10 @@
  		}
  	}
  
 -	if (isgraph(c) ||
 +	if ((flag & VIS_GLOB) &&
 +	    (c == '*' || c == '?' || c == '[' || c == '#'))
 +		;
 +	else if (isgraph(c) ||
  	   ((flag & VIS_SP) == 0 && c == ' ') ||
  	   ((flag & VIS_TAB) == 0 && c == '\t') ||
  	   ((flag & VIS_NL) == 0 && c == '\n') ||
 @@ -131,7 +134,7 @@
  			goto done;
  		}
  	}
 -	if (((c & 0177) == ' ') || (flag & VIS_OCTAL)) {
 +	if (((c & 0177) == ' ') || isgraph(c) || (flag & VIS_OCTAL)) {
  		*dst++ = '\\';
  		*dst++ = ((u_char)c >> 6 & 07) + '0';
  		*dst++ = ((u_char)c >> 3 & 07) + '0';
 @@ -163,7 +166,7 @@
   * strvis, strvisx - visually encode characters from src into dst
   *
   *	Dst must be 4 times the size of src to account for possible
 - *	expansion.  The length of dst, not including the trailing NULL,
 + *	expansion.  The length of dst, not including the trailing NUL,
   *	is returned.
   *
   *	Strvisx encodes exactly len bytes from src into dst.
 --- lib/libc/gen/vis.3.orig	Mon Apr 26 00:11:59 2004
 +++ lib/libc/gen/vis.3	Mon Apr 26 00:11:43 2004
 @@ -30,9 +30,9 @@
  .\" SUCH DAMAGE.
  .\"
  .\"     From: @(#)vis.3	8.1 (Berkeley) 6/9/93
  .\" $FreeBSD: src/lib/libc/gen/vis.3,v 1.8.2.6 2001/12/14 18:33:51 ru Exp $
  .\"
 -.Dd July 25, 1996
 +.Dd March 21, 2004
  .Dt VIS 3
  .Os
  .Sh NAME
 @@ -67,7 +67,9 @@
  encoding a set of characters into a buffer, the size of the buffer should
  be four times the number of characters encoded, plus one for the trailing
  .Dv NUL .
 -The flag parameter is used for altering the default range of
 +The
 +.Fa flag
 +argument is used for altering the default range of
  characters considered for encoding and for altering the visual
  representation.
  The additional character,
 @@ -135,6 +137,9 @@
  The following flags
  alter this:
  .Bl -tag -width VIS_WHITEX
 +.It Dv VIS_GLOB
 +Also encode magic characters ('*', '?', '[' and '#') recognized by
 +.Xr glob 3
  .It Dv VIS_SP
  Also encode space.
  .It Dv VIS_TAB
 @@ -227,7 +232,9 @@
  .Li \e0 Tn  - NUL No (000)
  .Ed
  .Pp
 -When using this format, the nextc parameter is looked at to determine
 +When using this format, the
 +.Fa nextc
 +argument is looked at to determine
  if a
  .Dv NUL
  character can be encoded as
 @@ -243,13 +250,13 @@
  The form is
  .Ql %dd
  where
 -.Em d
 +.Ar d
  represents a hexadecimal digit.
  .It Dv VIS_OCTAL
  Use a three digit octal sequence.  The form is
  .Ql \eddd
  where
 -.Em d
 +.Ar d
  represents an octal digit.
  .El
  .Pp
 @@ -275,3 +282,9 @@
  .Sh HISTORY
  These functions first appeared in
  .Bx 4.4 .
 +.Sh BUGS
 +The
 +.Nm
 +family of functions do not recognize multibyte characters, and thus
 +may consider them to be non-printable when they are in fact printable
 +(and vice versa.)
State-Changed-From-To: closed->patched 
State-Changed-By: kris 
State-Changed-When: Sun Apr 25 16:50:20 PDT 2004 
State-Changed-Why:  
Reopen the PR and assign to the author of the code in question, 
since the submitter claims it can in fact be MFCed safely. 

# If this is true I'd like to see it happen too. 


Responsible-Changed-From-To: freebsd-bugs->phk 
Responsible-Changed-By: kris 
Responsible-Changed-When: Sun Apr 25 16:50:20 PDT 2004 
Responsible-Changed-Why:  
Reopen the PR and assign to the author of the code in question, 
since the submitter claims it can in fact be MFCed safely. 

# If this is true I'd like to see it happen too. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60524 
Responsible-Changed-From-To: phk->simokawa 
Responsible-Changed-By: simokawa 
Responsible-Changed-When: Wed Aug 4 05:02:41 GMT 2004 
Responsible-Changed-Why:  
I'll take care of MFC. 

OK'ed by: phk via tamaru@myn.rcast.u-tokyo.ac.jp. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60524 
State-Changed-From-To: patched->feedback 
State-Changed-By: simokawa 
State-Changed-When: Fri Sep 17 05:44:39 GMT 2004 
State-Changed-Why:  
MFC done. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60524 
State-Changed-From-To: feedback->closed 
State-Changed-By: simokawa 
State-Changed-When: Wed Jan 12 21:49:49 GMT 2005 
State-Changed-Why:  
The submitter confirmed the change. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=60524 
>Unformatted:
