From nobody@FreeBSD.org  Tue Jan 16 02:44:12 2001
Return-Path: <nobody@FreeBSD.org>
Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id 7CB7C37B69B
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 16 Jan 2001 02:44:12 -0800 (PST)
Received: (from nobody@localhost)
	by freefall.freebsd.org (8.11.1/8.11.1) id f0GAiCb28930;
	Tue, 16 Jan 2001 02:44:12 -0800 (PST)
	(envelope-from nobody)
Message-Id: <200101161044.f0GAiCb28930@freefall.freebsd.org>
Date: Tue, 16 Jan 2001 02:44:12 -0800 (PST)
From: Jens.Schweikhardt@marconi.com
To: freebsd-gnats-submit@FreeBSD.org
Subject: make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
X-Send-Pr-Version: www-1.0

>Number:         24377
>Category:       bin
>Synopsis:       make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    schweikh
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 16 02:50:03 PST 2001
>Closed-Date:    Sat Jan 12 02:45:38 PST 2002
>Last-Modified:  Sat Jan 12 02:47:10 PST 2002
>Originator:     Jens Schweikhardt
>Release:        4.2-RELEASE
>Organization:
Marconi
>Environment:
FreeBSD bk4957 4.2-RELEASE FreeBSD 4.2-RELEASE #0: Mon Nov 20 13:02:55 GMT 2000     jkh@bento.FreeBSD.org:/usr/src/sys/compile/GENERIC  i386

>Description:
hello, world\n

It seems, make(1) doesn't understand substitutions in what the POSIX
standard calls internal variables ($@, $%, $?, $< and $*).
In the following example, the intent is to substitute any *.o in a *.c,
i.e. echo foo.c.

>How-To-Repeat:
$ cat Makefile                  
.POSIX:
foo.o:
        echo $(@F:.o=.c)
$ /usr/bin/make
echo foo.o.o=.c)
Syntax error: ")" unexpected
*** Error code 2

Stop in /usr/home/sje2bk.

GNU make works correctly, as well as Sun's /usr/ccs/bin/make:

$ /usr/local/bin/gmake
echo foo.c
foo.c
$                  


Is this a bug, a feature, lack of interest in POSIX conformance
or am I just confused? :-)

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->will 
Responsible-Changed-By: schweikh 
Responsible-Changed-When: Sat Jun 23 09:08:42 PDT 2001 
Responsible-Changed-Why:  
Over to maintainer. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=24377 
Responsible-Changed-From-To: will->freebsd-bugs 
Responsible-Changed-By: will 
Responsible-Changed-When: Wed Aug 29 16:25:07 PDT 2001 
Responsible-Changed-Why:  
I don't have time for make(1) anymore... 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=24377 

From: "Matthew Emmerton" <matt@gsicomp.on.ca>
To: <freebsd-gnats-submit@FreeBSD.org>,
	<Jens.Schweikhardt@marconi.com>
Cc:  
Subject: Re: bin/24377: make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
Date: Wed, 7 Nov 2001 23:27:04 -0500

 Here's a patch to the 4.3-REL version of make to fix this problem.  It's
 been tested with the example case in the PR.
 Basically, after expanding F or D, we fall-through to the expansion code,
 rather than returning right away.
 
 --- var.c.orig  Wed Nov  7 23:13:29 2001
 +++ var.c       Wed Nov  7 23:12:39 2001
 @@ -1622,8 +1622,6 @@
                         *freePtr = TRUE;
                         *lengthPtr = tstr-start+1;
                         *tstr = endc;
 -                       Buf_Destroy(buf, TRUE);
 -                       return(val);
                     }
                     break;
                 }
 
 --
 Matt Emmerton
 
State-Changed-From-To: open->analyzed 
State-Changed-By: schweikh 
State-Changed-When: Wed Nov 21 10:27:07 PST 2001 
State-Changed-Why:  
Analysis with patch received. 


Responsible-Changed-From-To: freebsd-bugs->schweikh 
Responsible-Changed-By: schweikh 
Responsible-Changed-When: Wed Nov 21 10:27:07 PST 2001 
Responsible-Changed-Why:  
I'm the originator and am going to test and integrate the provided patch. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=24377 

From: Jens Schweikhardt <schweikh@schweikhardt.net>
To: matt@gsicomp.on.ca
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/24377: make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
Date: Thu, 6 Dec 2001 22:14:01 +0100

 Matt,
 
 I've tried your patch on a -current system (applied manually, i.e.
 removed the two lines), but with the test case in the PR this dumps core:
 
 schweikh@hal9000:~ $ make bar.o
 bar.c
 make in free(): error: chunk is already free
 zsh: abort (core dumped)  make bar.o
 
 Can you reproduce this on a -current system? If it's hard for you to
 checkout from HEAD I can send you the current sources so you can have a
 look at it.
 
 Regards,
 
 	Jens
 -- 
 Jens Schweikhardt http://www.schweikhardt.net/
 SIGSIG -- signature too long (core dumped)

From: "Matthew Emmerton" <matt@gsicomp.on.ca>
To: <freebsd-gnats-submit@FreeBSD.org>,
	<Jens.Schweikhardt@marconi.com>
Cc:  
Subject: Re: bin/24377: make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
Date: Fri, 4 Jan 2002 00:39:38 -0500

 Here's a revised patch against 4.3-REL (silly mistake on my part) that
 should do the right thing.
 
 --- var.c.orig  Fri Jan  4 00:24:16 2002
 +++ var.c       Fri Jan  4 00:24:49 2002
 @@ -1614,15 +1614,8 @@
                         } else {
                             val = VarModify(val, VarTail, (ClientData)0);
                         }
 -                       /*
 -                        * Resulting string is dynamically allocated, so
 -                        * tell caller to free it.
 -                        */
 -                       *freePtr = TRUE;
                         *lengthPtr = tstr-start+1;
                         *tstr = endc;
 -                       Buf_Destroy(buf, TRUE);
 -                       return(val);
                     }
                     break;
                 }
 
 --
 Matt Emmerton
 

From: Jens Schweikhardt <schweikh@schweikhardt.net>
To: bug-followup@FreeBSD.org
Cc: matt@gsicomp.on.ca, mark@thuvia.demon.co.uk
Subject: Re: bin/24377: make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
Date: Fri, 11 Jan 2002 19:59:08 +0100

 Matt,
 
 your modified patch has a problem when no modifiers .x=.y are used:
 
 root@hal9000:/usr/obj/src/current/usr.bin/make # cat Makefile
 /tmp/xyz:
         echo $(@F)
 root@hal9000:/usr/obj/src/current/usr.bin/make # ./make /tmp/xyz
 echo /tmp/xyz
 /tmp/xyz
 
 It should echo xyz instead. Mark Valentine <mark@thuvia.demon.co.uk> has
 sent me a different patch, could you please have a look at it and tell me
 what you think about it?
 
 Index: var.c
 ===================================================================
 RCS file: /home/ncvs/src/usr.bin/make/var.c,v
 retrieving revision 1.21
 diff -u -r1.21 var.c
 --- var.c	2 Dec 2000 20:24:38 -0000	1.21
 +++ var.c	11 Jan 2002 18:40:29 -0000
 @@ -1599,7 +1599,7 @@
  		    vname[1] = '\0';
  		    v = VarFind(vname, ctxt, 0);
  
 -		    if (v != (Var *)NULL) {
 +		    if (v != (Var *)NULL && !haveModifier) {
  			/*
  			 * No need for nested expansion or anything, as we're
  			 * the only one who sets these things and we sure don't
 
 
 Regards,
 
 	Jens
 -- 
 Jens Schweikhardt http://www.schweikhardt.net/
 SIGSIG -- signature too long (core dumped)

From: "Matthew Emmerton" <matt@gsicomp.on.ca>
To: "Jens Schweikhardt" <schweikh@schweikhardt.net>,
	<bug-followup@FreeBSD.org>
Cc: <mark@thuvia.demon.co.uk>
Subject: Re: bin/24377: make(1) does not correctly substitute in internal macros like $(@F:.o=.c)
Date: Fri, 11 Jan 2002 23:44:54 -0500

 > Matt,
 > 
 > your modified patch has a problem when no modifiers .x=.y are used:
 > 
 > root@hal9000:/usr/obj/src/current/usr.bin/make # cat Makefile
 > /tmp/xyz:
 >         echo $(@F)
 > root@hal9000:/usr/obj/src/current/usr.bin/make # ./make /tmp/xyz
 > echo /tmp/xyz
 > /tmp/xyz
 > 
 > It should echo xyz instead. Mark Valentine <mark@thuvia.demon.co.uk> has
 > sent me a different patch, could you please have a look at it and tell me
 > what you think about it?
 > 
 > Index: var.c
 > ===================================================================
 > RCS file: /home/ncvs/src/usr.bin/make/var.c,v
 > retrieving revision 1.21
 > diff -u -r1.21 var.c
 > --- var.c 2 Dec 2000 20:24:38 -0000 1.21
 > +++ var.c 11 Jan 2002 18:40:29 -0000
 > @@ -1599,7 +1599,7 @@
 >       vname[1] = '\0';
 >       v = VarFind(vname, ctxt, 0);
 >  
 > -     if (v != (Var *)NULL) {
 > +     if (v != (Var *)NULL && !haveModifier) {
 >   /*
 >   * No need for nested expansion or anything, as we're
 >   * the only one who sets these things and we sure don't
 
 This patch looks like it does the right thing.
 
 --
 Matt Emmerton
 
State-Changed-From-To: analyzed->closed 
State-Changed-By: schweikh 
State-Changed-When: Sat Jan 12 02:45:38 PST 2002 
State-Changed-Why:  
Applied the "&& !haveModifier" one-line patch to -current. Thanks Mark! 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=24377 
>Unformatted:

On Sat, Jun 23, 2001 at 05:47:04PM +0000, Giorgos Keramidas wrote:
# On Tue, Jan 16, 2001 at 02:44:12AM -0800, Jens.Schweikhardt@marconi.com wrote:
#
# > >Description:
# >
# > It seems, make(1) doesn't understand substitutions in what the POSIX
# > standard calls internal variables ($@, $%, $?, $< and $*).
# > In the following example, the intent is to substitute any *.o in a *.c,
# > i.e. echo foo.c.
# >
# > >How-To-Repeat:
# > $ cat Makefile
# > .POSIX:
# > foo.o:
# >         echo $(@F:.o=.c)
# > $ /usr/bin/make
# > echo foo.o.o=.c)
# > Syntax error: ")" unexpected
# > *** Error code 2
# >
# > Stop in /usr/home/sje2bk.
#
# Jens, are you sure this problem still persists to bug you?
  
Yes.

# Well, the syntax is slightly different in the make(1) program I have
# on my 5.0-CURRENT, and by looking at the CVS logs, it seems that its
# been like that for a while.  The way that BSD make does substitution
# in variables, special or not is described in the paper found at
# /usr/share/doc/psd/12.make/paper.* and an example of it is shown
# below:
  
That document also says:
  
  In addition, the System V style of substitution is also sup-
  ported.  This looks like:

        $(VARIABLE:search-string=replacement)

and I can see nothing that says I couldn't use @F for VARIABLE here.

...
# Am afraid that I don't know if POSIX specifies a different syntax than
# the ${VARIABLE:modifier} that BSD make uses.

I've got the old (1995) and soon-to-be (2001) POSIX specs near me.
They are very clear. I know of the workaround; but I'm interested in
having /usr/bin/make be POSIX compliant. There are makefiles out there
that use this feature and they fail on BSD for no good reason.
The latest and final POSIX draft says;

 Macro expansions using the forms $(string1[:subst1=[subst2]]) or
 ${string1[:subst1=[subst2]]} can be used to replace all occurrences of
 subst1 with subst2 when the macro substitution is performed. The subst1
 to be replaced shall be recognized when it is a suffix at the end of a
 word in string1 (where a word, in this context, is defined to be a
 string delimited by the beginning of the line, a <blank> or <newline>).
 If string1 in a macro expansion contains a macro expansion, the results
 are unspecified. Macro expansions in string1 of macro definition lines
 shall be evaluated when read. Macro expansions in string2 of macro
 definition lines shall be performed when the macro identified by
 string1 is expanded in a rule or command.

#  On the other hand, the
# version of GNU make that I have says in its manuals that substitution
# in variables works a little differently than what you wrote.

Not really; the % way of doing substitution is a GNU extension.
It's not POSIX. It's just another (albeit more flexible) way of doing
substitution. And as you can see in my original PR, GNU gets the POSIX
substitution right.

#       % cat GNUmakefile
#
#       SRCS=   main.c foo.c
#
#       OBJS=   $(SRCS:%.c=%.o)
#
#       all:
#               echo $(OBJS)
#
#       % gmake -f GNUmakefile
#
#       echo main.o foo.o
#       main.o foo.o
#
# I think we can close this PR, if thats ok with you Jens too.

Please leave it open. This is a real bug. POSIX substitution
works in other circumstance with regular variables, just not with
automatics using D (directory part) and F (filename part) modifiers:

.POSIX:
FOO = bar.o

all:
    @echo '$(FOO:.o=.c)'
bar.o:
    @echo '$(@F:.o=.c)'
zap.o:
    @echo '$(@:.o=.c)'


$ make all
bar.c                  # works
$ make bar.o
bar.o.o=.c)            # buggy, should be bar.c
$ make zap.o
zap.c                  # works

Regards,

        Jens

