From nobody@FreeBSD.org  Tue Jul  9 10:38:02 2002
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 0861E37B400
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  9 Jul 2002 10:38:02 -0700 (PDT)
Received: from www.freebsd.org (www.FreeBSD.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id C277A43E09
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  9 Jul 2002 10:38:01 -0700 (PDT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.4/8.12.4) with ESMTP id g69Hc1OT046274
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 9 Jul 2002 10:38:01 -0700 (PDT)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.4/8.12.4/Submit) id g69Hc1i0046273;
	Tue, 9 Jul 2002 10:38:01 -0700 (PDT)
Message-Id: <200207091738.g69Hc1i0046273@www.freebsd.org>
Date: Tue, 9 Jul 2002 10:38:01 -0700 (PDT)
From: Chet Ramey <chet@po.cwru.edu>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Parsing problem with /bin/sh
X-Send-Pr-Version: www-1.0

>Number:         40386
>Category:       bin
>Synopsis:       Parsing problem with /bin/sh
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    tjr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jul 09 10:40:02 PDT 2002
>Closed-Date:    Sun Aug 04 01:26:37 PDT 2002
>Last-Modified:  Sat Aug 10 20:10:02 PDT 2002
>Originator:     Chet Ramey
>Release:        4.6-RELEASE, 5.0-CURRENT
>Organization:
Case Western Reserve University
>Environment:
FreeBSD quartz.ins.cwru.edu 5.0-CURRENT FreeBSD 5.0-CURRENT #3: Tue Jul  2 09:13:32 EDT 2002     chet@quartz.ins.cwru.edu:/usr/obj/usr/src/sys/CNS-OPTIPLEX  i386

>Description:
     There is a parsing problem with /bin/sh.  The attached script should not work without a backslash at the end of the first line, but sh accepts it nevertheless.

This faulty syntax is used in /usr/libexec/locate.mklocatedb
(/usr/src/usr.bin/locate/locate/mklocatedb.sh).
>How-To-Repeat:
quartz(2)# cat x1
awk '{print 12345}' </dev/null
        || exit 1
quartz(2)# ./sh ./x1
quartz(2)# /bin/bash ./x1
./x1: line 2: syntax error near unexpected token `||'
./x1: line 2: ` || exit 1'


>Fix:
The patch for mklocatedb.sh is obvious; lines 79 and 87.


>Release-Note:
>Audit-Trail:

From: Giorgos Keramidas <keramida@FreeBSD.org>
To: Chet Ramey <chet@po.cwru.edu>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Wed, 10 Jul 2002 22:54:25 +0300

 On 2002-07-09 10:38 +0000, Chet Ramey wrote:
 > There is a parsing problem with /bin/sh.  The attached script should
 > not work without a backslash at the end of the first line, but sh
 > accepts it nevertheless.
 
 > quartz(2)# cat x1
 > awk '{print 12345}' </dev/null
 >         || exit 1
 > quartz(2)# ./sh ./x1
 > quartz(2)# /bin/bash ./x1
 > ./x1: line 2: syntax error near unexpected token `||'
 > ./x1: line 2: ` || exit 1'
 
 Just to clarify a bit.  There is nothing wrong with sh(1) here.  The
 first line is not continued to the second one with a backslash, and it
 runs as expected.  The second line is faulty though, because sh(1)
 attempts to parse it as a complete command and it (correctly) fails.
 
 > This faulty syntax is used in /usr/libexec/locate.mklocatedb
 > (/usr/src/usr.bin/locate/locate/mklocatedb.sh).
 
 This is a problem :/
 How does the following change look?
 
 %%%
 Index: mklocatedb.sh
 ===================================================================
 RCS file: /home/ncvs/src/usr.bin/locate/locate/mklocatedb.sh,v
 retrieving revision 1.12
 diff -u -r1.12 mklocatedb.sh
 --- mklocatedb.sh	23 Jun 2002 00:57:17 -0000	1.12
 +++ mklocatedb.sh	10 Jul 2002 19:53:20 -0000
 @@ -75,17 +75,17 @@
      # This scheme avoid large temporary files in /tmp
  
      $code $bigrams > $filelist || exit 1
 -    locate -d $filelist / | $bigram | $sort -nr | head -128 |
 -    awk '{if (/^[ 	]*[0-9]+[ 	]+..$/) {printf("%s",$2)} else {exit 1}}' > $bigrams
 -	|| exit 1
 +    locate -d $filelist / | $bigram | $sort -nr | head -128 | \
 +    awk '{if (/^[ 	]*[0-9]+[ 	]+..$/) {printf("%s",$2)} else {exit 1}}' \
 +	> $bigrams || exit 1
      locate -d $filelist / | $code $bigrams || exit 1
      exit 	
  
  else
      if $sortcmd $sortopt > $filelist; then
 -        $bigram < $filelist | $sort -nr | 
 -	awk '{if (/^[ 	]*[0-9]+[ 	]+..$/) {printf("%s",$2)} else {exit 1}}' > $bigrams
 -	    || exit 1
 +        $bigram < $filelist | $sort -nr | \
 +	awk '{if (/^[ 	]*[0-9]+[ 	]+..$/) {printf("%s",$2)} else {exit 1}}' \
 +	    > $bigrams || exit 1
          $code $bigrams < $filelist || exit 1
      else
          echo "`basename $0`: cannot build locate database" >&2
 %%%

From: Chet Ramey <chet@nike.ins.cwru.edu>
To: keramida@FreeBSD.org
Cc: chet@po.cwru.edu, bug-followup@FreeBSD.org
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Wed, 10 Jul 2002 16:27:18 -0400

 > On 2002-07-09 10:38 +0000, Chet Ramey wrote:
 > > There is a parsing problem with /bin/sh.  The attached script should
 > > not work without a backslash at the end of the first line, but sh
 > > accepts it nevertheless.
 > 
 > > quartz(2)# cat x1
 > > awk '{print 12345}' </dev/null
 > >         || exit 1
 > > quartz(2)# ./sh ./x1
 > > quartz(2)# /bin/bash ./x1
 > > ./x1: line 2: syntax error near unexpected token `||'
 > > ./x1: line 2: ` || exit 1'
 > 
 > Just to clarify a bit.  There is nothing wrong with sh(1) here.  The
 > first line is not continued to the second one with a backslash, and it
 > runs as expected.  The second line is faulty though, because sh(1)
 > attempts to parse it as a complete command and it (correctly) fails.
 
 I think you misread the report.  sh accepts the script without error,
 when it clearly should not.  /bin/bash is the one that flags the error,
 and is included as an example of correct behavior.
 
 I used `./sh' from /usr/obj/usr/src/bin/sh so I could be sure that I had
 the latest version I cvsup'd.
 
 Chet
 
 -- 
 ``The lyf so short, the craft so long to lerne.'' - Chaucer
 ( ``Discere est Dolere'' -- chet )
 
 Chet Ramey, CWRU    chet@po.CWRU.Edu    http://cnswww.cns.cwru.edu/~chet/
Responsible-Changed-From-To: freebsd-bugs->tjr 
Responsible-Changed-By: tjr 
Responsible-Changed-When: Sun Jul 21 22:31:37 PDT 2002 
Responsible-Changed-Why:  
I have a patch ready for this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40386 
State-Changed-From-To: open->patched 
State-Changed-By: tjr 
State-Changed-When: Sun Jul 21 22:50:31 PDT 2002 
State-Changed-Why:  
mklocatedb and sh have been fixed, I will MFC the changes after 
two weeks. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40386 
State-Changed-From-To: patched->closed 
State-Changed-By: tjr 
State-Changed-When: Sun Aug 4 01:25:46 PDT 2002 
State-Changed-Why:  
Change has been MFC'd, thanks for pointing this out. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40386 

From: The Anarcat <anarcat@anarcat.ath.cx>
To: freebsd-gnats-submit@FreeBSD.org, chet@po.cwru.edu
Cc: tjr@FreeBSD.org
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Tue, 6 Aug 2002 19:21:02 -0400

 Well, it seems that mklocatedb hasn't been completely fixed:
 
 A recent post on -stable outlines the problem, which is:
 
 "
 =46rom /usr/libexec/locate.mklocatedb line 87:                  =20
                                                                 =20
 perl -ne '/^\s*[0-9]+\s(..)$/ && print $1 || exit 1' > $bigrams            =
  =20
         || exit 1                                             =20
 "
 
 (Message-ID: <Pine.BSF.4.44.0208061621470.93936-100000@elara.frii.com>)
 
 The fix is:
 
 --- locate.mklocatedb.old	Tue Aug  6 19:18:53 2002
 +++ locate.mklocatedb	Tue Aug  6 19:19:16 2002
 @@ -83,8 +83,7 @@
  else
      if $sortcmd $sortopt > $filelist; then
          $bigram < $filelist | $sort -nr |=20
 -	perl -ne '/^\s*[0-9]+\s(..)$/ && print $1 || exit 1' > $bigrams=20
 -	    || exit 1
 +	perl -ne '/^\s*[0-9]+\s(..)$/ && print $1 || exit 1' > $bigrams || exit 1
          $code $bigrams < $filelist || exit 1
      else
          echo "`basename $0`: cannot build locate database" >&2
 
 A few other utilities are broken by this change, the mysqld startup
 script being the only one I could find for now. I'll contact the mysql
 maintainer for that one.
 
 A.
 
 --=20
 Un =E9ducateur dans l'=E2me ne prend rien au s=E9rieux que par rapport
 =E0 ses disciples -- soi-m=EAme non except=E9.
                         - Nietzsche, "Par del=E0 le bien et le mal"

From: Tim Robbins <tjr@FreeBSD.ORG>
To: The Anarcat <anarcat@anarcat.ath.cx>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Wed, 7 Aug 2002 19:22:11 +1000

 On Tue, Aug 06, 2002 at 07:21:02PM -0400, The Anarcat wrote:
 
 > Well, it seems that mklocatedb hasn't been completely fixed:
 > 
 > A recent post on -stable outlines the problem, which is:
 > 
 > "
 > >From /usr/libexec/locate.mklocatedb line 87:                   
 >                                                                  
 > perl -ne '/^\s*[0-9]+\s(..)$/ && print $1 || exit 1' > $bigrams              
 >         || exit 1                                              
 > "
 > 
 > (Message-ID: <Pine.BSF.4.44.0208061621470.93936-100000@elara.frii.com>)
 > 
 > The fix is:
 > 
 > --- locate.mklocatedb.old	Tue Aug  6 19:18:53 2002
 > +++ locate.mklocatedb	Tue Aug  6 19:19:16 2002
 > @@ -83,8 +83,7 @@
 >  else
 >      if $sortcmd $sortopt > $filelist; then
 >          $bigram < $filelist | $sort -nr | 
 > -	perl -ne '/^\s*[0-9]+\s(..)$/ && print $1 || exit 1' > $bigrams 
 > -	    || exit 1
 > +	perl -ne '/^\s*[0-9]+\s(..)$/ && print $1 || exit 1' > $bigrams || exit 1
 >          $code $bigrams < $filelist || exit 1
 >      else
 >          echo "`basename $0`: cannot build locate database" >&2
 
 Sorry, I forgot to update the script in the STABLE branch when I merged
 the shell change across. I've committed your patch now.
 
 > A few other utilities are broken by this change, the mysqld startup
 > script being the only one I could find for now. I'll contact the mysql
 > maintainer for that one.
 
 Constructs like:
 foo
 || echo foo failed
 
 were never equivalent to:
 foo || echo foo failed
 
 The latter is interpreted as:
 foo
 true || echo foo failed
 
 ie. the stuff after the || is never executed.
 
 Thanks for pointing this problem out.
 
 
 Tim

From: Antoine Beaupre <anarcat@anarcat.ath.cx>
To: Tim Robbins <tjr@FreeBSD.ORG>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Wed, 7 Aug 2002 10:43:46 -0400

 On Wednesday, August 7, 2002, at 05:22 AM, Tim Robbins wrote:
 >
 > Sorry, I forgot to update the script in the STABLE branch when I merged
 > the shell change across. I've committed your patch now.
 >
 > Thanks for pointing this problem out.
 
 My pleasure!
 
 A.
 

From: FUJISHIMA Satsuki <sf@FreeBSD.org>
To: FreeBSD-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Sun, 11 Aug 2002 04:41:16 +0900

 I don't know these statements follow standard but that break the case:
 
 --- test.sh ---
 if :; then :; fi
 > /nonexistent || echo "can't create /nonexistent"
 --- test.sh ---
 
 $ for s in bash pdksh zsh sh; do echo $s; $s test.sh; done
 bash
 test.sh: line 2: /nonexistent: Permission denied
 can't create test.txt
 pdksh
 test.sh[2]: cannot create /nonexistent: Permission denied
 can't create test.txt
 zsh
 test.sh:2: permission denied: /nonexistent
 can't create test.txt
 sh
 test.sh: 2: Syntax error: "||" unexpected
 $ 

From: Tim Robbins <tjr@FreeBSD.org>
To: FUJISHIMA Satsuki <sf@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Sun, 11 Aug 2002 12:23:28 +1000

 On Sat, Aug 10, 2002 at 12:50:04PM -0700, FUJISHIMA Satsuki wrote:
 
 >  I don't know these statements follow standard but that break the case:
 >  
 >  --- test.sh ---
 >  if :; then :; fi
 >  > /nonexistent || echo "can't create /nonexistent"
 >  --- test.sh ---
 
 This has never worked with FreeBSD's /bin/sh:
 
 $ > /nonexistent || echo "can't create /nonexistent"
 cannot create /nonexistent: permission denied
 [the `echo' command is never executed]
 $ uname -r
 4.6.1-RELEASE-p10
 
 I'll check the standards and see whether they say this construct ought
 to work.
 
 
 Tim

From: Tim Robbins <tjr@FreeBSD.org>
To: FUJISHIMA Satsuki <sf@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: bin/40386: Parsing problem with /bin/sh
Date: Sun, 11 Aug 2002 13:06:34 +1000

 On Sat, Aug 10, 2002 at 12:50:04PM -0700, FUJISHIMA Satsuki wrote:
 
 > The following reply was made to PR bin/40386; it has been noted by GNATS.
 > 
 > From: FUJISHIMA Satsuki <sf@FreeBSD.org>
 > To: FreeBSD-gnats-submit@FreeBSD.org
 > Cc:  
 > Subject: Re: bin/40386: Parsing problem with /bin/sh
 > Date: Sun, 11 Aug 2002 04:41:16 +0900
 > 
 >  I don't know these statements follow standard but that break the case:
 >  
 >  --- test.sh ---
 >  if :; then :; fi
 >  > /nonexistent || echo "can't create /nonexistent"
 >  --- test.sh ---
 
 Thanks, you were right. parser.c revision 1.42 fixes this, and allows
 	>/dev/null && echo foo
 while disallowing
 	&& echo foo
 
 I'll MFC it in about a week; definitely before RELENG_4 is frozen for 4.7.
 Thanks!
 
 
 Tim
>Unformatted:
