From yar@FreeBSD.org  Tue Aug 11 07:06:34 2009
Return-Path: <yar@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 6D4911065694
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 11 Aug 2009 07:06:34 +0000 (UTC)
	(envelope-from yar@FreeBSD.org)
Received: from ref8-amd64.freebsd.org (ref8-amd64.freebsd.org [IPv6:2001:4f8:fff6::5f])
	by mx1.freebsd.org (Postfix) with ESMTP id 5D12E8FC1F
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 11 Aug 2009 07:06:34 +0000 (UTC)
Received: from ref8-amd64.freebsd.org (localhost [127.0.0.1])
	by ref8-amd64.freebsd.org (8.14.3/8.14.3) with ESMTP id n7B76Yhr041940
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 11 Aug 2009 07:06:34 GMT
	(envelope-from yar@ref8-amd64.freebsd.org)
Received: (from yar@localhost)
	by ref8-amd64.freebsd.org (8.14.3/8.14.3/Submit) id n7B76YDu041939;
	Tue, 11 Aug 2009 07:06:34 GMT
	(envelope-from yar)
Message-Id: <200908110706.n7B76YDu041939@ref8-amd64.freebsd.org>
Date: Tue, 11 Aug 2009 07:06:34 GMT
From: Yar Tikhiy <yar@freebsd.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: sh(1): /bin/sh misses unmatched quotes in backticks
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         137657
>Category:       bin
>Synopsis:       sh(1): /bin/sh misses unmatched quotes in backticks
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    jilles
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Aug 11 07:10:03 UTC 2009
>Closed-Date:    Sun Oct 04 16:40:30 UTC 2009
>Last-Modified:  Sun Oct 04 16:40:30 UTC 2009
>Originator:     Yar Tikhiy
>Release:        FreeBSD 8.0-BETA2 amd64
>Organization:
BarNet
>Environment:
System: FreeBSD ref8-amd64.freebsd.org 8.0-BETA2 FreeBSD 8.0-BETA2 #11 r195838: Fri Jul 24 09:52:42 UTC 2009 simon@ref8-amd64.freebsd.org:/scratch/src/sys/amd64/compile/REF8-AMD64 amd64
>Description:

/bin/sh misses an unmatched quote in backticks.  The closing
backtick will implicitly close the open quote as well.  This
can lead to latent bugs in shell scripts.

>How-To-Repeat:

$ echo `echo '123`
123
$ echo `echo "123`
123

>Fix:
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->jilles 
Responsible-Changed-By: jilles 
Responsible-Changed-When: Fri Aug 21 14:16:10 UTC 2009 
Responsible-Changed-Why:  
Take 

http://www.freebsd.org/cgi/query-pr.cgi?pr=137657 

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, yar@freebsd.org
Cc:  
Subject: Re: bin/137657: sh(1): /bin/sh misses unmatched quotes in backticks
Date: Fri, 21 Aug 2009 16:11:01 +0200

 Originally, this behaviour was not a bug, but designed in, probably in
 imitation of System V sh and/or various versions of real ksh. Removing
 it (treating unmatched quotes in backticks as errors) simplifies the
 code (removing the parsebackquote variable entirely).
 
 Considering that bash, zsh, pdksh, NetBSD /bin/sh and dash also treat
 unmatched quotes in backticks as errors and that it is good in principle
 to detect as many errors as possible at parse time, I think it is
 appropriate to fix this (after head thaws).
 
 -- 
 Jilles Tjoelker

From: Yar Tikhiy <yar.tikhiy@gmail.com>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@freebsd.org
Subject: Re: bin/137657: sh(1): /bin/sh misses unmatched quotes in backticks
Date: Wed, 26 Aug 2009 09:18:28 +1000

 Hi Jilles,
 
 On Sat, Aug 22, 2009 at 12:11 AM, Jilles Tjoelker<jilles@stack.nl> wrote:
 > Originally, this behaviour was not a bug, but designed in, probably in
 > imitation of System V sh and/or various versions of real ksh. Removing
 > it (treating unmatched quotes in backticks as errors) simplifies the
 > code (removing the parsebackquote variable entirely).
 
 Thank you so much for putting this into historical perspective! It
 hasn't occurred to me it's a compatibility shim, not a genuine ash
 bug.
 
 > Considering that bash, zsh, pdksh, NetBSD /bin/sh and dash also treat
 > unmatched quotes in backticks as errors and that it is good in principle
 > to detect as many errors as possible at parse time, I think it is
 > appropriate to fix this (after head thaws).
 
 You seem to know how to fix this, right?  I'll greatly appreciate if
 you find a free minute to make a preliminary patch.
 
 And maybe you can have some idea about bin/137659 (another backtick
 related bug in ash) as well?
 
 Thanks a lot!
 
 -- 
 Yar

From: Jilles Tjoelker <jilles@stack.nl>
To: Yar Tikhiy <yar.tikhiy@gmail.com>
Cc: bug-followup@freebsd.org
Subject: Re: bin/137657: sh(1): /bin/sh misses unmatched quotes in backticks
Date: Sat, 29 Aug 2009 18:18:26 +0200

 --PNTmBPCT7hxwcZjr
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 On Wed, Aug 26, 2009 at 09:18:28AM +1000, Yar Tikhiy wrote:
 > On Sat, Aug 22, 2009 at 12:11 AM, Jilles Tjoelker<jilles@stack.nl> wrote:
 > > Considering that bash, zsh, pdksh, NetBSD /bin/sh and dash also treat
 > > unmatched quotes in backticks as errors and that it is good in principle
 > > to detect as many errors as possible at parse time, I think it is
 > > appropriate to fix this (after head thaws).
 
 > You seem to know how to fix this, right?  I'll greatly appreciate if
 > you find a free minute to make a preliminary patch.
 
 Attached is a patch with testcase.
 
 In case gnats mangles the patch, the important point is just to remove
 the  ! parsebackquote &&  thing from parser.c, the rest just removes a
 now unnecessary variable.
 
 -- 
 Jilles Tjoelker
 
 --PNTmBPCT7hxwcZjr
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="backquote-quote-mismatch.patch"
 
 Index: bin/sh/parser.c
 ===================================================================
 --- bin/sh/parser.c	(revision 196614)
 +++ bin/sh/parser.c	(working copy)
 @@ -82,7 +82,6 @@
  
  
  STATIC struct heredoc *heredoclist;	/* list of here documents to read */
 -STATIC int parsebackquote;	/* nonzero if we are inside backquotes */
  STATIC int doprompt;		/* if set, prompt the user */
  STATIC int needprompt;		/* true if interactive and at start of line */
  STATIC int lasttoken;		/* last token read */
 @@ -1043,7 +1042,7 @@
  endword:
  	if (syntax == ARISYNTAX)
  		synerror("Missing '))'");
 -	if (syntax != BASESYNTAX && ! parsebackquote && eofmark == NULL)
 +	if (syntax != BASESYNTAX && eofmark == NULL)
  		synerror("Unterminated quoted string");
  	if (varnest != 0) {
  		startlinno = plinno;
 @@ -1303,7 +1302,6 @@
  
  parsebackq: {
  	struct nodelist **nlpp;
 -	int savepbq;
  	union node *n;
  	char *volatile str;
  	struct jmploc jmploc;
 @@ -1311,11 +1309,9 @@
  	int savelen;
  	int saveprompt;
  
 -	savepbq = parsebackquote;
  	if (setjmp(jmploc.loc)) {
  		if (str)
  			ckfree(str);
 -		parsebackquote = 0;
  		handler = savehandler;
  		longjmp(handler->loc, 1);
  	}
 @@ -1397,7 +1393,6 @@
  		nlpp = &(*nlpp)->next;
  	*nlpp = (struct nodelist *)stalloc(sizeof (struct nodelist));
  	(*nlpp)->next = NULL;
 -	parsebackquote = oldstyle;
  
  	if (oldstyle) {
  		saveprompt = doprompt;
 @@ -1433,7 +1428,6 @@
  		str = NULL;
  		INTON;
  	}
 -	parsebackquote = savepbq;
  	handler = savehandler;
  	if (arinest || dblquote)
  		USTPUTC(CTLBACKQ | CTLQUOTE, out);
 Index: tools/regression/bin/sh/errors/backquote-error2.0
 ===================================================================
 --- tools/regression/bin/sh/errors/backquote-error2.0	(revision 0)
 +++ tools/regression/bin/sh/errors/backquote-error2.0	(revision 0)
 @@ -0,0 +1,7 @@
 +# $FreeBSD$
 +
 +sh -c 'echo `echo .BA"DCODE.`
 +echo ".BAD"CODE.' 2>&1 | grep -q BADCODE && exit 1
 +echo '`"`' | sh -n 2>/dev/null || exit 1
 +echo '`'"'"'`' | sh -n 2>/dev/null || exit 1
 +exit 0
 
 Property changes on: tools/regression/bin/sh/errors/backquote-error2.0
 ___________________________________________________________________
 Added: svn:keywords
    + FreeBSD=%H
 
 
 --PNTmBPCT7hxwcZjr--

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/137657: commit references a PR
Date: Thu,  1 Oct 2009 21:40:28 +0000 (UTC)

 Author: jilles
 Date: Thu Oct  1 21:40:08 2009
 New Revision: 197691
 URL: http://svn.freebsd.org/changeset/base/197691
 
 Log:
   sh: Disallow mismatched quotes in backticks (`...`).
   
   Due to the amount of code removed by this, it seems that allowing unmatched
   quotes was a deliberate imitation of System V sh and real ksh. Most other
   shells do not allow unmatched quotes (e.g. bash, zsh, pdksh, NetBSD /bin/sh,
   dash).
   
   PR:		bin/137657
 
 Added:
   head/tools/regression/bin/sh/errors/backquote-error2.0   (contents, props changed)
 Modified:
   head/bin/sh/parser.c
 
 Modified: head/bin/sh/parser.c
 ==============================================================================
 --- head/bin/sh/parser.c	Thu Oct  1 21:14:40 2009	(r197690)
 +++ head/bin/sh/parser.c	Thu Oct  1 21:40:08 2009	(r197691)
 @@ -82,7 +82,6 @@ struct heredoc {
  
  
  STATIC struct heredoc *heredoclist;	/* list of here documents to read */
 -STATIC int parsebackquote;	/* nonzero if we are inside backquotes */
  STATIC int doprompt;		/* if set, prompt the user */
  STATIC int needprompt;		/* true if interactive and at start of line */
  STATIC int lasttoken;		/* last token read */
 @@ -1043,7 +1042,7 @@ readtoken1(int firstc, char const *synta
  endword:
  	if (syntax == ARISYNTAX)
  		synerror("Missing '))'");
 -	if (syntax != BASESYNTAX && ! parsebackquote && eofmark == NULL)
 +	if (syntax != BASESYNTAX && eofmark == NULL)
  		synerror("Unterminated quoted string");
  	if (varnest != 0) {
  		startlinno = plinno;
 @@ -1303,7 +1302,6 @@ parsesub: {
  
  parsebackq: {
  	struct nodelist **nlpp;
 -	int savepbq;
  	union node *n;
  	char *volatile str;
  	struct jmploc jmploc;
 @@ -1311,11 +1309,9 @@ parsebackq: {
  	int savelen;
  	int saveprompt;
  
 -	savepbq = parsebackquote;
  	if (setjmp(jmploc.loc)) {
  		if (str)
  			ckfree(str);
 -		parsebackquote = 0;
  		handler = savehandler;
  		longjmp(handler->loc, 1);
  	}
 @@ -1397,7 +1393,6 @@ done:
  		nlpp = &(*nlpp)->next;
  	*nlpp = (struct nodelist *)stalloc(sizeof (struct nodelist));
  	(*nlpp)->next = NULL;
 -	parsebackquote = oldstyle;
  
  	if (oldstyle) {
  		saveprompt = doprompt;
 @@ -1433,7 +1428,6 @@ done:
  		str = NULL;
  		INTON;
  	}
 -	parsebackquote = savepbq;
  	handler = savehandler;
  	if (arinest || dblquote)
  		USTPUTC(CTLBACKQ | CTLQUOTE, out);
 
 Added: head/tools/regression/bin/sh/errors/backquote-error2.0
 ==============================================================================
 --- /dev/null	00:00:00 1970	(empty, because file is newly added)
 +++ head/tools/regression/bin/sh/errors/backquote-error2.0	Thu Oct  1 21:40:08 2009	(r197691)
 @@ -0,0 +1,7 @@
 +# $FreeBSD$
 +
 +sh -c 'echo `echo .BA"DCODE.`
 +echo ".BAD"CODE.' 2>&1 | grep -q BADCODE && exit 1
 +echo '`"`' | sh -n 2>/dev/null && exit 1
 +echo '`'"'"'`' | sh -n 2>/dev/null && exit 1
 +exit 0
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: jilles 
State-Changed-When: Sun Oct 4 16:40:29 UTC 2009 
State-Changed-Why:  
Fixed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=137657 
>Unformatted:
