From val@sqdp.trc-net.co.jp  Mon Mar  5 03:35:13 2001
Return-Path: <val@sqdp.trc-net.co.jp>
Received: from wg.pu.ru (wg.pu.ru [193.124.85.219])
	by hub.freebsd.org (Postfix) with ESMTP id 7A17637B719
	for <FreeBSD-gnats-submit@freebsd.org>; Mon,  5 Mar 2001 03:35:11 -0800 (PST)
	(envelope-from val@sqdp.trc-net.co.jp)
Received: from sqdp.trc-net.co.jp (p90.ppp.pu.ru [194.85.123.90])
	by wg.pu.ru (8.9.1a/8.9.1) with ESMTP id LAA25410
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 5 Mar 2001 11:35:02 GMT
Received: (from val@localhost)
	by sqdp.trc-net.co.jp (8.9.3/8.9.3) id OAA05323;
	Mon, 5 Mar 2001 14:39:53 +0300 (MSK)
	(envelope-from val)
Message-Id: <200103051139.OAA05323@sqdp.trc-net.co.jp>
Date: Mon, 5 Mar 2001 14:39:53 +0300 (MSK)
From: val@sqdp.trc-net.co.jp
Reply-To: val@sqdp.trc-net.co.jp
To: FreeBSD-gnats-submit@freebsd.org
Subject: Undocumented bug in /bin/sh
X-Send-Pr-Version: 3.2

>Number:         25542
>Category:       bin
>Synopsis:       sh(1) null char in quoted string
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 05 03:40:01 PST 2001
>Closed-Date:    Sun Jan 17 14:13:40 UTC 2010
>Last-Modified:  Sun Jan 17 14:20:01 UTC 2010
>Originator:     Valentin Davydov
>Release:        FreeBSD 4.1-RELEASE i386
>Organization:
State Optical Institute
>Environment:
Most typical environments for users and scripts who use /bin/sh.
Especially dangerous when these scripts take external parameters.

>Description:
/bin/sh improperly handles null characters within quoted strings.
This behaviour is not documented in the man page, where the only
char prohibited within single-quoted string is said to be the single 
quote itself.

>How-To-Repeat:
The command

echo -e "echo \047O\000K\047" | /bin/sh

yields only three bytes on stdout: "O", "K" and <LF>, whereas it must
produce four of them: "O", <NULL>, "K" and <LF>, as the command

echo -e 'O\000K'

does. Another example is given  by the following gawk script (utilizing
the fact that Gnu awk calls /bin/sh to interpret the external system
commands):

#!/usr/bin/awk -f
BEGIN{print system("echo 'O\000K'")}

>Fix:
Avoid usage of the null(or whatever else hardcoded)-terminated strings 
in programs which deal with character strings. See awk or perl for the 
implementation details.

Workaround: explicity state in the documentation that <NULL> character 
is prohibited for use in the shell parameters.
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: gnats-admin->freebsd-bugs 
Responsible-Changed-By: phk 
Responsible-Changed-When: Wed Apr 4 00:38:57 PDT 2001 
Responsible-Changed-Why:  
over to bugs 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25336 
Responsible-Changed-From-To: gnats-admin->freebsd-bugs 
Responsible-Changed-By: phk 
Responsible-Changed-When: Wed Apr 4 00:38:57 PDT 2001 
Responsible-Changed-Why:  
over to bugs 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25346 
Responsible-Changed-From-To: gnats-admin->freebsd-bugs 
Responsible-Changed-By: phk 
Responsible-Changed-When: Wed Apr 4 00:38:57 PDT 2001 
Responsible-Changed-Why:  
over to bugs 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 
Responsible-Changed-From-To: freebsd-bugs->standards 
Responsible-Changed-By: schweikh 
Responsible-Changed-When: Thu Aug 22 13:05:16 PDT 2002 
Responsible-Changed-Why:  
This might be standards@ fodder. Are NULs in this way 
defined/undefined behavior? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 
Responsible-Changed-From-To: standards->freebsd-standards 
Responsible-Changed-By: des 
Responsible-Changed-When: Tue Feb 17 07:45:16 PST 2004 
Responsible-Changed-Why:  
Misattributed PR. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 

From: Volker <volker@vwsoft.com>
To: bug-followup@FreeBSD.org, val@sqdp.trc-net.co.jp
Cc:  
Subject: Re: bin/25542: /bin/sh: null char in quoted string
Date: Sat, 26 Jan 2008 12:42:16 +0100

 I've checked this on 6.2-REL and 7.0-PRE using:
 
 echo -e "\047O\000K\047" | od -c
 
 Result:
 
 0000000    '   O  \0   K   '  \n
 0000006
 
 This is what I expect to come out and it shows, sh is handling NULL
 bytes correctly ATM.
 
 Please follow-up with your opinion , otherwise this PR can be closed.
State-Changed-From-To: open->feedback 
State-Changed-By: keramida 
State-Changed-When: Sat Jan 26 11:51:46 UTC 2008 
State-Changed-Why:  
Set to feedback. volker@vwsoft.com cannot reproduce this with 
recent builds of 6.X and 7.X, so if this is not a problem anymore 
we can close the ticket. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 
State-Changed-From-To: feedback->closed 
State-Changed-By: rwatson 
State-Changed-When: Sun Jan 27 13:56:32 UTC 2008 
State-Changed-Why:  
Close as submitter e-mail address bounces.  If you are the submitter 
and want to re-open this PR, please follow up with a new e-mail 
address so that the PR can be updated.  Please also include further 
debugging information as we cannot reproduce the probelm on more 
recent FreeBSD versions.  Thanks for the report! 


http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 
State-Changed-From-To: closed->open 
State-Changed-By: gavin 
State-Changed-When: Sun Jan 27 14:31:11 UTC 2008 
State-Changed-Why:  
Reopen, this can still be recreated on 7.0: 

$ echo -e "echo \047O\000K\047" | /bin/sh | od -c 
0000000    O   K  n 
0000003 


http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/25542: sh(1) null char in quoted string
Date: Sat, 4 Apr 2009 14:41:43 +0200

 Considering that fixing this would be a lot of work and cannot be done
 completely (for example, argument strings and environment variables
 cannot contain '\0'), I think it is best to close this. sh(1) is meant
 to process text, not binary data. Trying to process binary data may or
 will also cause problems if the locale character set is set to UTF-8.
 
 As a clarification, this PR is about '\0' bytes in shell scripts, not
 about making the echo builtin produce '\0' characters. The latter
 feature works fine and is good.
 
 -- 
 Jilles Tjoelker

From: Valentin Davydov <sp@m.davydov.spb.su>
To: bug-followup@FreeBSD.org, sp@m.davydov.spb.su
Cc: Jilles Tjoelker <jilles@stack.nl>
Subject: Re: bin/25542: sh(1) null char in quoted string
Date: Mon, 27 Jul 2009 22:50:54 +0400 (MSD)

 At Sat, 4 Apr 2009 14:41:43 +0200, Jilles Tjoelker wrote:
 >Considering that fixing this would be a lot of work and cannot be done
 >completely (for example, argument strings and environment variables
 >cannot contain '\0'), I think it is best to close this.
 
 I think, at least documentation issue mentioned in the original PR 25542
 can be corrected easy. Here is the patch:
 
 --- src/bin/sh/sh.1.orig        2007-12-05 17:29:07.000000000 +0300
 +++ src/bin/sh/sh.1     2009-07-27 22:36:39.000000000 +0400
 @@ -2381,4 +2381,6 @@
  .Sh BUGS
  The
  .Nm
 -utility does not recognize multibyte characters.
 +utility does not recognize multibyte characters.
 +ASCII <NULL> character in input strings, parameters etc. can be mishandled by
 +.Nm .
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/25542: commit references a PR
Date: Sun, 20 Sep 2009 21:42:52 +0000 (UTC)

 Author: jilles
 Date: Sun Sep 20 21:42:38 2009
 New Revision: 197371
 URL: http://svn.freebsd.org/changeset/base/197371
 
 Log:
   Mention that NUL characters are not allowed in sh(1) input.
   
   I do not consider this a bug because POSIX permits it and argument strings
   and environment variables cannot contain '\0' anyway.
   
   PR:		bin/25542
   MFC after:	2 weeks
 
 Modified:
   head/bin/sh/sh.1
 
 Modified: head/bin/sh/sh.1
 ==============================================================================
 --- head/bin/sh/sh.1	Sun Sep 20 19:32:10 2009	(r197370)
 +++ head/bin/sh/sh.1	Sun Sep 20 21:42:38 2009	(r197371)
 @@ -375,6 +375,10 @@ introduces a comment if used at the begi
  The word starting with
  .Ql #
  and the rest of the line are ignored.
 +.Pp
 +.Tn ASCII
 +.Dv NUL
 +characters (character code 0) are not allowed in shell input.
  .Ss Quoting
  Quoting is used to remove the special meaning of certain characters
  or words to the shell, such as operators, whitespace, keywords,
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
Responsible-Changed-From-To: freebsd-standards->jilles 
Responsible-Changed-By: jilles 
Responsible-Changed-When: Sun Oct 4 17:15:40 UTC 2009 
Responsible-Changed-Why:  
Take. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/25542: commit references a PR
Date: Sun,  4 Oct 2009 17:16:22 +0000 (UTC)

 Author: jilles
 Date: Sun Oct  4 17:16:11 2009
 New Revision: 197748
 URL: http://svn.freebsd.org/changeset/base/197748
 
 Log:
   MFC r197371: Mention that NUL characters are not allowed in sh(1) input.
   
   I do not consider this a bug because POSIX permits it and argument strings
   and environment variables cannot contain '\0' anyway.
   
   PR:		bin/25542
 
 Modified:
   stable/7/bin/sh/   (props changed)
   stable/7/bin/sh/sh.1
 
 Modified: stable/7/bin/sh/sh.1
 ==============================================================================
 --- stable/7/bin/sh/sh.1	Sun Oct  4 16:30:33 2009	(r197747)
 +++ stable/7/bin/sh/sh.1	Sun Oct  4 17:16:11 2009	(r197748)
 @@ -375,6 +375,10 @@ introduces a comment if used at the begi
  The word starting with
  .Ql #
  and the rest of the line are ignored.
 +.Pp
 +.Tn ASCII
 +.Dv NUL
 +characters (character code 0) are not allowed in shell input.
  .Ss Quoting
  Quoting is used to remove the special meaning of certain characters
  or words to the shell, such as operators, whitespace, keywords,
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: jilles 
State-Changed-When: Sun Oct 4 17:29:59 UTC 2009 
State-Changed-Why:  
I will merge this to stable/8 after 8.0 release. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/25542: commit references a PR
Date: Thu, 29 Oct 2009 21:14:09 +0000 (UTC)

 Author: jilles
 Date: Thu Oct 29 21:13:57 2009
 New Revision: 198624
 URL: http://svn.freebsd.org/changeset/base/198624
 
 Log:
   MFC r197371: Mention that NUL characters are not allowed in sh(1) input.
   
   I do not consider this a bug because POSIX permits it and argument strings
   and environment variables cannot contain '\0' anyway.
   
   PR:		bin/25542
 
 Modified:
   stable/8/bin/sh/   (props changed)
   stable/8/bin/sh/sh.1
 
 Modified: stable/8/bin/sh/sh.1
 ==============================================================================
 --- stable/8/bin/sh/sh.1	Thu Oct 29 20:53:26 2009	(r198623)
 +++ stable/8/bin/sh/sh.1	Thu Oct 29 21:13:57 2009	(r198624)
 @@ -375,6 +375,10 @@ introduces a comment if used at the begi
  The word starting with
  .Ql #
  and the rest of the line are ignored.
 +.Pp
 +.Tn ASCII
 +.Dv NUL
 +characters (character code 0) are not allowed in shell input.
  .Ss Quoting
  Quoting is used to remove the special meaning of certain characters
  or words to the shell, such as operators, whitespace, keywords,
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
Responsible-Changed-From-To: jilles->freebsd-bugs 
Responsible-Changed-By: jilles 
Responsible-Changed-When: Sun Nov 8 16:55:24 UTC 2009 
Responsible-Changed-Why:  
I'm not interested in 6.x. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 
State-Changed-From-To: patched->closed 
State-Changed-By: emaste 
State-Changed-When: Sun Jan 17 14:11:56 UTC 2010 
State-Changed-Why:  
Previously merged to 8 and 7, and I've merged to 6 now too for completeness. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=25542 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/25542: commit references a PR
Date: Sun, 17 Jan 2010 14:12:02 +0000 (UTC)

 Author: emaste
 Date: Sun Jan 17 14:11:42 2010
 New Revision: 202484
 URL: http://svn.freebsd.org/changeset/base/202484
 
 Log:
   Log:
   MFC r197371: Mention that NUL characters are not allowed in sh(1) input.
   
   PR: bin/25542
 
 Modified:
   stable/6/bin/sh/sh.1
 Directory Properties:
   stable/6/bin/sh/   (props changed)
 
 Modified: stable/6/bin/sh/sh.1
 ==============================================================================
 --- stable/6/bin/sh/sh.1	Sun Jan 17 13:49:18 2010	(r202483)
 +++ stable/6/bin/sh/sh.1	Sun Jan 17 14:11:42 2010	(r202484)
 @@ -370,6 +370,10 @@ introduces a comment if used at the begi
  The word starting with
  .Ql #
  and the rest of the line are ignored.
 +.Pp
 +.Tn ASCII
 +.Dv NUL
 +characters (character code 0) are not allowed in shell input.
  .Ss Quoting
  Quoting is used to remove the special meaning of certain characters
  or words to the shell, such as operators, whitespace, keywords,
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
