From rfg@tristatelogic.com  Wed Jun 27 21:41:13 2012
Return-Path: <rfg@tristatelogic.com>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 51AA3106564A
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 27 Jun 2012 21:41:13 +0000 (UTC)
	(envelope-from rfg@tristatelogic.com)
Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118])
	by mx1.freebsd.org (Postfix) with ESMTP id 29E308FC16
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 27 Jun 2012 21:41:13 +0000 (UTC)
Received: by segfault.tristatelogic.com (Postfix, from userid 1237)
	id A1EA35081B; Wed, 27 Jun 2012 14:41:05 -0700 (PDT)
Message-Id: <20120627214105.A1EA35081B@segfault.tristatelogic.com>
Date: Wed, 27 Jun 2012 14:41:05 -0700 (PDT)
From: Ronald F.Guilmette <rfg@tristatelogic.com>
Reply-To: Ronald F.Guilmette <rfg@tristatelogic.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: /bin/expr improperly requires forward slash to be escaped
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         169500
>Category:       bin
>Synopsis:       expr(1) improperly requires forward slash to be escaped
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-standards
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jun 27 21:50:01 UTC 2012
>Closed-Date:    Thu Jun 28 17:22:45 UTC 2012
>Last-Modified:  Thu Jun 28 22:20:09 UTC 2012
>Originator:     Ronald F. Guilmette
>Release:        FreeBSD 8.3-RELEASE amd64
>Organization:
entropy
>Environment:

8.3-RELEASE amd64

>Description:

According to my reading of Sections 2.8.3.1.1 and 2.8.3.1.2 of the ANSI/IEEE
standard 1003.2, a forward slash character (/) is an "ordinary" basic
regular expression (BRE) character, and as such can be used in a BRE alone
to match itself.  As far as I can tell, backslash-escaping of this specific
character should not be required within BREs.

I note however that /bin/expr does not allow a naked (non-escaped) forward
slash character within the right-hand operand of the : binary operator.  For
example, /bin/expr complains about a "syntax" error on either of the following
commands:
	
	/bin/expr abcxyz : /
	/bin/expr abcxyz : "/"

I believe that these complaints on the part of /bin/expr are unfounded and
inappropriate, and furthermore that they are not consistant with the 1003.2
standard.

In contrast, the following command does not produce a syntax error:

	/bin/expr abcxyz : "\/"

This fact leads me to believe that the FreeBSD implementation of /bin/expr
may perhaps be improperly treating forward slash as some sort of a delimiter
within the BRE operand of the : binary operator.

In my opinion, this should be corrected in order to insure maximal compliance
with the 1003.2 POSIX standard.

***  Please note also that my tests and this report apply strictly and only
to /bin/expr.  However "expr" may perhaps be implemented as a built-in in
some or all shells (e.g. sh, bash, csh, tcsh) and thus, my hope is that
whoever attends to this PR will also take the time to look into the possibility
that fixes are needed also in those shells.

>How-To-Repeat:

	/bin/expr abcxyz : /
	/bin/expr abcxyz : "/"

>Fix:

I haven't looked at the code, so I cannot propose a fix.  As I have said above
however, my guess is that /bin/expr is in some way treating the right-hand
(BRE) operand of the : binary operator as a thing that either is, or that
will be, at some point in the internal processing, delimited (on both the
left and right) with forward slashes.  If so, then /bin/expr should take care
to pre-escape any user-supplied forward slashes within the user-supplied
BRE operand.
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Jun 27 23:52:26 UTC 2012 
Responsible-Changed-Why:  
fix Synopsis. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=169500 

From: Garrett Wollman <wollman@hergotha.csail.mit.edu>
To: rfg@tristatelogic.com
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: bin/169500: /bin/expr improperly requires forward slash to be
	escaped
Date: Wed, 27 Jun 2012 21:36:24 -0400 (EDT)

 In article <20120627214105.A1EA35081B@segfault.tristatelogic.com>,
 rfg@tristatelogic.com writes:
 
 >According to my reading of Sections 2.8.3.1.1 and 2.8.3.1.2 of the ANSI/IEEE
 >standard 1003.2, a forward slash character (/) is an "ordinary" basic
 >regular expression (BRE) character, and as such can be used in a BRE alone
 >to match itself.
 
 IEEE Std.1003.2 was obsoleted by the publication of IEEE
 Std.1003.1-2001; your reference is at least fifteen years old.
 
 >As far as I can tell, backslash-escaping of this specific character
 >should not be required within BREs.
 
 Nothing to do with the BRE-ness.  As the 2001 edition of the standard
 states (page 429, lines 16598-16599):
 
 	A string argument is an argument that cannot be identified as
 	an integer argument or as one of the expression operator
 	symbols shown in the OPERANDS section.
 
 '/' is "one of the expression operator symbols shown", and thus is not
 permitted as the second (or first!) operand of the ':' operator, which
 must both be "string arguments" meeting this defintion.  As the
 APPLICATION USAGE section states (same page, lines 16609ff):
 
 	After argument processing by the shell, expr is not required
 	to be able to tell the difference between an operator and an
 	operand except by the value. If "$a" is '=', the command:
 
 	    expr $a = '='
 
 	looks like:
 
 	    expr = = =
 
 	as the arguments are passed to expr (and they all may be taken
 	as the '=' operator).
 
 -GAWollman
 
 -- 
 Garrett A. Wollman    | What intellectual phenomenon can be older, or more oft
 wollman@bimajority.org| repeated, than the story of a large research program
 Opinions not shared by| that impaled itself upon a false central assumption
 my employers.         | accepted by all practitioners? - S.J. Gould, 1993

From: Mark Linimon <linimon@lonesome.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/169500: expr(1) improperly requires forward slash to be
 escaped
Date: Wed, 27 Jun 2012 21:24:40 -0500

 ----- Forwarded message from Robert Bonomi <bonomi@mail.r-bonomi.com> -----
 
 Date: Wed, 27 Jun 2012 20:44:25 -0500 (CDT)
 From: Robert Bonomi <bonomi@mail.r-bonomi.com>
 To: linimon@freebsd.org
 Subject: Re: bin/169500: expr(1) improperly requires forward slash to be escaped
 
 FWIW, the putative 'bug' is documented on the manpage for expr.
 
 There is nothing wrong with the 1003 regex handling.
 
 It is operator/operaand recognition in expr.  Using something that
 is parsable as an operator _as_ an operand. is a syntax error for 
 the ':' operator -- EXPRESSLY so stated on the manpage. 
 Using _any_ token that is parsable as an 'operator' (arithmetic, or
 logical) as an operand for ':' will result in the  same non-bug error
 
 The 'Examples' section of the manpage documents a work-around --
 actually using '/' as the example.
 
 Shoulc be closable -- with the traditional IBMism -- "it's not a bug,
 it's a _feature_, and =documented= as such."    *BIG* grin.
 
 ----- End forwarded message -----
Responsible-Changed-From-To: freebsd-bugs->freebsd-standards 
Responsible-Changed-By: eadler 
Responsible-Changed-When: Thu Jun 28 06:25:59 UTC 2012 
Responsible-Changed-Why:  
over to you 

http://www.freebsd.org/cgi/query-pr.cgi?pr=169500 
State-Changed-From-To: open->closed 
State-Changed-By: wollman 
State-Changed-When: Thu Jun 28 17:21:35 UTC 2012 
State-Changed-Why:  
expr(1) behaves as the Standard says it is supposed to.  '/' is an operator 
and thus not parsed as a possible string value. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=169500 

From: "Ronald F. Guilmette" <rfg@tristatelogic.com>
To: Garrett Wollman <wollman@hergotha.csail.mit.edu>
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: bin/169500: /bin/expr improperly requires forward slash to be escaped
Date: Thu, 28 Jun 2012 15:11:55 -0700

 In message <201206280136.q5S1aOQO093902@hergotha.csail.mit.edu>, you wrote:
 
 >In article <20120627214105.A1EA35081B@segfault.tristatelogic.com>,
 >rfg@tristatelogic.com writes:
 >
 >>According to my reading of Sections 2.8.3.1.1 and 2.8.3.1.2 of the ANSI/IEEE
 >>standard 1003.2, a forward slash character (/) is an "ordinary" basic
 >>regular expression (BRE) character, and as such can be used in a BRE alone
 >>to match itself.
 >
 >IEEE Std.1003.2 was obsoleted by the publication of IEEE
 >Std.1003.1-2001; your reference is at least fifteen years old.
 
 Yes.  My bad.  Sorry.
 
 (These publications are quite expensive, as you may know, and thus, I have
 not updated mine in quite some time.)
 
 >>As far as I can tell, backslash-escaping of this specific character
 >>should not be required within BREs.
 >
 >Nothing to do with the BRE-ness.  As the 2001 edition of the standard
 >states (page 429, lines 16598-16599):
 >
 >	A string argument is an argument that cannot be identified as
 >	an integer argument or as one of the expression operator
 >	symbols shown in the OPERANDS section.
 
 Humm... OK.  How do I retract my PR?
 
>Unformatted:
