From ed@hoeg.nl  Tue Jun 19 17:49:41 2007
Return-Path: <ed@hoeg.nl>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id D07CF16A46B
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 19 Jun 2007 17:49:41 +0000 (UTC)
	(envelope-from ed@hoeg.nl)
Received: from palm.hoeg.nl (mx0.hoeg.nl [83.98.131.211])
	by mx1.freebsd.org (Postfix) with ESMTP id 9C9B513C483
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 19 Jun 2007 17:49:41 +0000 (UTC)
	(envelope-from ed@hoeg.nl)
Received: by palm.hoeg.nl (Postfix, from userid 1000)
	id D3EFC1CE25; Tue, 19 Jun 2007 19:49:39 +0200 (CEST)
Message-Id: <20070619174939.D3EFC1CE25@palm.hoeg.nl>
Date: Tue, 19 Jun 2007 19:49:39 +0200 (CEST)
From: Ed Schouten <ed@fxq.nl>
Reply-To: Ed Schouten <ed@fxq.nl>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: sh: shell is still running when using `sh -c'
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         113860
>Category:       bin
>Synopsis:       sh(1): shell is still running when using `sh -c'
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jun 19 17:50:09 GMT 2007
>Closed-Date:    Sat Jun 13 21:46:38 UTC 2009
>Last-Modified:  Sat Jun 13 21:46:38 UTC 2009
>Originator:     Ed Schouten
>Release:        FreeBSD 6.2-STABLE i386
>Organization:
>Environment:
System: FreeBSD palm.hoeg.nl 6.2-STABLE FreeBSD 6.2-STABLE #0: Fri Apr 20 13:44:49 CEST 2007 root@palm.hoeg.nl:/usr/obj/usr/src/sys/PALM i386
>Description:
Some bloated shell from GNU immediately shuts down the shell when using
`sh -c'. This means that:

sh -c 'sleep 10'

only keeps a sleep(1) process running for 10 seconds; the shell is
closed immediately.

FreeBSD's sh doesn't. This means that all applications that use
system(3) or just execve(2) /bin/sh leave shells running around.
>How-To-Repeat:
| $ sh -c 'sleep 10' &
| [1] 13519
| $ ps ux
| USER   PID %CPU %MEM   VSZ   RSS  TT  STAT STARTED      TIME COMMAND
| ...
| ed   13519  0.0  0.2  6228  1808  p2  SN    7:48PM   0:00.00 sh -c sleep 10
| ed   13520  0.0  0.1  2552   704  p2  SN    7:48PM   0:00.00 sleep 10
>Fix:
I looked through the sh source code, but I can't seem to find a way to
do this. I'm not an expert when it comes to patching sh(1) code. :-(
>Release-Note:
>Audit-Trail:

From: Oliver Fromme <olli@lurza.secnetix.de>
To: freebsd-bugs@FreeBSD.ORG, Ed Schouten <ed@fxq.nl>,
        bug-followup@FreeBSD.ORG
Cc:  
Subject: Re: bin/113860: sh: shell is still running when using `sh -c'
Date: Tue, 19 Jun 2007 20:29:36 +0200 (CEST)

 Ed Schouten wrote:
  > Some bloated shell from GNU immediately shuts down the shell when using
  > `sh -c'. This means that:
  > 
  > sh -c 'sleep 10'
  > 
  > only keeps a sleep(1) process running for 10 seconds; the shell is
  > closed immediately.
 
 That's only possible for simple cases, i.e. when a single
 external command is used.  In all other cases, the shell
 process has to be kept, e.g.:
 
 sh -c 'sleep 10; echo foobar'
 
 In that case the shell process has to stay and wait for the
 command to finish.
 
  > FreeBSD's sh doesn't. This means that all applications that use
  > system(3) or just execve(2) /bin/sh leave shells running around.
 
 Which is not a big deal.  Those shell processes are quite
 cheap.  Most of the RSS that you see in top is shared
 between processes, so the RAM consumption is negligible.
 
 If you want to avoid the shell process anyway, a simple
 workaround is to preped "exec":
 
 sh -c 'exec sleep 10'
 
 In that case the shell will exec() the program directly,
 instead of fork() and waiting for it to finish.  Actually
 that's exactly what that "bloated shell from GNU" does
 automatically (prepending "exec" internally).
 
 Implementing that automatic behaviour in FreeBSD's /bin/sh
 is not trivial, because you have to find out first if it
 is possible for the given command line or not.
 
 Apart from that I'm not sure if it is desirable to have
 such a "feature".  Personally I believe that the shell
 should do what I tell it to do.  That is, when I want it
 to use "exec", I write "exec".  If I don't want it, I
 don't write it.  I would be somewhat surprised if it used
 "exec" when I don't instruct it to do so.  It might be
 regarded as a POLA violation, but it's probably a matter
 of opinion.
 
 Best regards
    Oliver
 
 -- 
 Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
 Handelsregister: Registergericht Muenchen, HRA 74606,  Geschftsfuehrung:
 secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mn-
 chen, HRB 125758,  Geschftsfhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart
 
 FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd
 
 $ dd if=/dev/urandom of=test.pl count=1
 $ file test.pl
 test.pl: perl script text executable

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, ed@freebsd.org, olli@lurza.secnetix.de,
	freebsd-hackers@freebsd.org
Cc:  
Subject: Re: bin/113860: sh(1): shell is still running when using `sh -c'
Date: Fri, 3 Apr 2009 23:39:05 +0200

 I think this can be improved.
 
 Given that I've been digging in /bin/sh already...
 
 Note first that sh already has some of this functionality:
 
 % sh -c '{ echo a; sleep 10;}&'; sleep 1; ps T
 a
   PID  TT  STAT      TIME COMMAND
 94682  p9  Ss     0:00.07 zsh
 94702  p9  S      0:00.00 sleep 10
 94704  p9  R+     0:00.00 ps T
 %
 
 This is the EV_EXIT flag to evaltree() and friends, in eval.c.
 
 To make this work for '-c', evalstring() needs some flag like EV_EXIT,
 and parsecmd() needs to tell evalstring() that the command it read is
 the last (currently, parsecmd() only reports that there is no command
 anymore; due to the stack-like memory management it is not really
 possible to read ahead a command). Putting "{\n" and "\n}" around the
 string could be an alternative for the latter, as any valid string would
 consist of one (compound) command only.
 
 The new mode for evalstring() would only be used for '-c' commands when
 '-s' is not given.
 
 Apart from bash, ksh93 and Solaris /usr/xpg4/bin/sh (which is basically
 ksh88) also treat simple commands in '-c' this way. So I think the idea
 is ok. I'm also slightly annoyed by seeing silly 'sh -c blah' processes
 hanging around, and it is not always possible or desirable to add
 'exec'.
 
 On another note, the EV_EXIT mode is erroneously still used if a trap on
 EXIT has been set (or, maybe, any trap at all; particularly if -T is in
 effect). This means that such traps may not be executed. Most other
 shells seem to do this right.
 
 -- 
 Jilles Tjoelker

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, ed@freebsd.org, olli@lurza.secnetix.de,
	freebsd-hackers@freebsd.org
Cc:  
Subject: Re: bin/113860: sh(1): shell is still running when using `sh -c'
Date: Sun, 12 Apr 2009 23:33:14 +0200

 --9amGYk9869ThD9tj
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 On Fri, Apr 03, 2009 at 11:39:05PM +0200, Jilles Tjoelker wrote:
 > I think this can be improved.
 
 I have a patch now; it does not handle all cases but the effect on the
 code is minimal. I decided to go with the readahead but only do it for
 the first line.
 
 To avoid problems with traps not being executed,
 http://www.stack.nl/~jilles/unix/sh-forkiftrapped.patch is needed. This
 fixes a bug in EV_EXIT handling, which would be triggered more often
 with the change to -c. The patch is also needed for bin/74404.
 Example: sh -c '(trap "echo trapped" EXIT; sleep 3)'
 
 http://www.stack.nl/~jilles/unix/sh-minusc-exec.patch the change itself
 
 -- 
 Jilles Tjoelker
 
 --9amGYk9869ThD9tj
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="sh-forkiftrapped.patch"
 
 Don't skip forking for an external command if any traps are active.
 
 Example:
   sh -c '(trap "echo trapped" EXIT; sleep 3)'
 now correctly prints "trapped".
 
 With this check, it is no longer necessary to check for -T
 explicitly in that case.
 
 diff --git a/eval.c b/eval.c
 --- a/eval.c
 +++ b/eval.c
 @@ -730,7 +730,7 @@ evalcommand(union node *cmd, int flags, 
  	/* Fork off a child process if necessary. */
  	if (cmd->ncmd.backgnd
  	 || (cmdentry.cmdtype == CMDNORMAL
 -	    && ((flags & EV_EXIT) == 0 || Tflag))
 +	    && ((flags & EV_EXIT) == 0 || have_traps()))
  	 || ((flags & EV_BACKCMD) != 0
  	    && (cmdentry.cmdtype != CMDBUILTIN
  		 || cmdentry.u.index == CDCMD
 diff --git a/trap.c b/trap.c
 --- a/trap.c
 +++ b/trap.c
 @@ -222,6 +222,21 @@ clear_traps(void)
  
  
  /*
 + * Check if we have any traps enabled.
 + */
 +int
 +have_traps(void)
 +{
 +	char *volatile *tp;
 +
 +	for (tp = trap ; tp <= &trap[NSIG - 1] ; tp++) {
 +		if (*tp && **tp)	/* trap not NULL or SIG_IGN */
 +			return 1;
 +	}
 +	return 0;
 +}
 +
 +/*
   * Set the signal handler for the specified signal.  The routine figures
   * out what it should be set to.
   */
 diff --git a/trap.h b/trap.h
 --- a/trap.h
 +++ b/trap.h
 @@ -39,6 +39,7 @@ extern volatile sig_atomic_t gotwinch;
  
  int trapcmd(int, char **);
  void clear_traps(void);
 +int have_traps(void);
  void setsignal(int);
  void ignoresig(int);
  void onsig(int);
 
 --9amGYk9869ThD9tj
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="sh-minusc-exec.patch"
 
 Avoid leaving unnecessary waiting shells in many forms of sh -c COMMAND.
 
 This change only affects strings passed to -c, when the -s
 option is not used.
 
 The approach is to read the first line of commands, then the
 second, and if it turns out there is no second line execute
 the first line with EV_EXIT. Otherwise execute the first and
 second line, then read and execute lines as long as they
 exist.
 
 A compound statement such as introduced by {, if, for or
 while counts as a single line of commands for this.
 
 Note that if the second line contains a syntactical error,
 the first line will not be executed, different from
 previously. (pdksh and zsh parse the entire -c string
 before executing it.)
 
 Example:
   sh -c 'ps lT'
 No longer shows a shell process waiting for ps to finish.
 
 diff --git a/eval.c b/eval.c
 --- a/eval.c
 +++ b/eval.c
 @@ -175,6 +175,44 @@ evalstring(char *s)
  }
  
  
 +/*
 + * Like evalstring(), but always exits. This is useful in avoiding shell
 + * processes just sitting around waiting for exit.
 + */
 +
 +void
 +evalstring_exit(char *s)
 +{
 +	union node *n, *n2;
 +	struct stackmark smark;
 +
 +	setstackmark(&smark);
 +	setinputstring(s, 1);
 +	do
 +		n = parsecmd(0);
 +	while (n == NULL);
 +	if (n != NEOF) {
 +		do
 +			n2 = parsecmd(0);
 +		while (n2 == NULL);
 +		if (n2 == NEOF) {
 +			evaltree(n, EV_EXIT);
 +			/*NOTREACHED*/
 +		}
 +		evaltree(n, 0);
 +		evaltree(n2, 0);
 +		popstackmark(&smark);
 +		while ((n = parsecmd(0)) != NEOF) {
 +			if (n != NULL)
 +				evaltree(n, 0);
 +			popstackmark(&smark);
 +		}
 +	}
 +	popfile();
 +	popstackmark(&smark);
 +	exitshell(exitstatus);
 +}
 +
  
  /*
   * Evaluate a parse tree.  The value is left in the global variable
 diff --git a/eval.h b/eval.h
 --- a/eval.h
 +++ b/eval.h
 @@ -47,6 +47,7 @@ struct backcmd {		/* result of evalbackc
  
  int evalcmd(int, char **);
  void evalstring(char *);
 +void evalstring_exit(char *);
  union node;	/* BLETCH for ansi C */
  void evaltree(union node *, int);
  void evalbackcmd(union node *, struct backcmd *);
 diff --git a/main.c b/main.c
 --- a/main.c
 +++ b/main.c
 @@ -178,7 +178,10 @@ state2:
  state3:
  	state = 4;
  	if (minusc) {
 -		evalstring(minusc);
 +		if (sflag)
 +			evalstring(minusc);
 +		else
 +			evalstring_exit(minusc);
  	}
  	if (sflag || minusc == NULL) {
  state4:	/* XXX ??? - why isn't this before the "if" statement */
 
 --9amGYk9869ThD9tj--

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, ed@fxq.nl
Cc:  
Subject: Re: bin/113860: sh(1): shell is still running when using `sh -c'
Date: Thu, 11 Jun 2009 23:49:35 +0200

 Although it's probably too late for the slush/freeze, here is an updated
 version of sh-minusc-exec.patch:
 - the old one did not apply anymore
 - new approach that does not change the parsing anymore, instead
   checking if we are at the end of the string
 
 http://www.stack.nl/~jilles/unix/sh-minusc-exec-2.patch
 
 Note, sh-forkiftrapped.patch is still needed, and not changed.
 
 -- 
 Jilles Tjoelker

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/113860: commit references a PR
Date: Sat, 13 Jun 2009 21:17:56 +0000 (UTC)

 Author: jilles
 Date: Sat Jun 13 21:17:45 2009
 New Revision: 194128
 URL: http://svn.freebsd.org/changeset/base/194128
 
 Log:
   Avoid leaving unnecessary waiting shells in many forms of sh -c COMMAND.
   
   This change only affects strings passed to -c, when the -s
   option is not used.
   
   The approach is to check if there may be additional data
   in the string after parsing each command. If there is none,
   use the EV_EXIT flag so that a fork may be omitted in
   specific cases.
   
   If there are empty lines after the command, the check will
   not see the end and forks will not be omitted. The same
   thing seems to happen in bash.
   
   Example:
     sh -c 'ps lT'
   No longer shows a shell process waiting for ps to finish.
   
   PR:		bin/113860
   Reviewed by:	stefanf
   Approved by:	ed (mentor)
 
 Modified:
   head/bin/sh/eval.c
   head/bin/sh/eval.h
   head/bin/sh/input.c
   head/bin/sh/input.h
   head/bin/sh/main.c
 
 Modified: head/bin/sh/eval.c
 ==============================================================================
 --- head/bin/sh/eval.c	Sat Jun 13 21:10:41 2009	(r194127)
 +++ head/bin/sh/eval.c	Sat Jun 13 21:17:45 2009	(r194128)
 @@ -74,11 +74,6 @@ __FBSDID("$FreeBSD$");
  #endif
  
  
 -/* flags in argument to evaltree */
 -#define EV_EXIT 01		/* exit after evaluating tree */
 -#define EV_TESTED 02		/* exit status is checked; ignore -e flag */
 -#define EV_BACKCMD 04		/* command executing within back quotes */
 -
  MKINIT int evalskip;		/* set if we are skipping commands */
  STATIC int skipcount;		/* number of levels to skip */
  MKINIT int loopnest;		/* current loop nesting level */
 @@ -163,20 +158,28 @@ evalstring(char *s, int flags)
  {
  	union node *n;
  	struct stackmark smark;
 +	int flags_exit;
  
 +	flags_exit = flags & EV_EXIT;
 +	flags &= ~EV_EXIT;
  	setstackmark(&smark);
  	setinputstring(s, 1);
  	while ((n = parsecmd(0)) != NEOF) {
 -		if (n != NULL)
 -			evaltree(n, flags);
 +		if (n != NULL) {
 +			if (flags_exit && preadateof())
 +				evaltree(n, flags | EV_EXIT);
 +			else
 +				evaltree(n, flags);
 +		}
  		popstackmark(&smark);
  	}
  	popfile();
  	popstackmark(&smark);
 +	if (flags_exit)
 +		exitshell(exitstatus);
  }
  
  
 -
  /*
   * Evaluate a parse tree.  The value is left in the global variable
   * exitstatus.
 
 Modified: head/bin/sh/eval.h
 ==============================================================================
 --- head/bin/sh/eval.h	Sat Jun 13 21:10:41 2009	(r194127)
 +++ head/bin/sh/eval.h	Sat Jun 13 21:17:45 2009	(r194128)
 @@ -45,6 +45,11 @@ struct backcmd {		/* result of evalbackc
  	struct job *jp;		/* job structure for command */
  };
  
 +/* flags in argument to evaltree/evalstring */
 +#define EV_EXIT 01		/* exit after evaluating tree */
 +#define EV_TESTED 02		/* exit status is checked; ignore -e flag */
 +#define EV_BACKCMD 04		/* command executing within back quotes */
 +
  int evalcmd(int, char **);
  void evalstring(char *, int);
  union node;	/* BLETCH for ansi C */
 
 Modified: head/bin/sh/input.c
 ==============================================================================
 --- head/bin/sh/input.c	Sat Jun 13 21:10:41 2009	(r194127)
 +++ head/bin/sh/input.c	Sat Jun 13 21:17:45 2009	(r194128)
 @@ -321,6 +321,23 @@ check:
  }
  
  /*
 + * Returns if we are certain we are at EOF. Does not cause any more input
 + * to be read from the outside world.
 + */
 +
 +int
 +preadateof(void)
 +{
 +	if (parsenleft > 0)
 +		return 0;
 +	if (parsefile->strpush)
 +		return 0;
 +	if (parsenleft == EOF_NLEFT || parsefile->buf == NULL)
 +		return 1;
 +	return 0;
 +}
 +
 +/*
   * Undo the last call to pgetc.  Only one character may be pushed back.
   * PEOF may be pushed back.
   */
 
 Modified: head/bin/sh/input.h
 ==============================================================================
 --- head/bin/sh/input.h	Sat Jun 13 21:10:41 2009	(r194127)
 +++ head/bin/sh/input.h	Sat Jun 13 21:17:45 2009	(r194128)
 @@ -48,6 +48,7 @@ extern int init_editline;	/* 0 == not se
  char *pfgets(char *, int);
  int pgetc(void);
  int preadbuffer(void);
 +int preadateof(void);
  void pungetc(void);
  void pushstring(char *, int, void *);
  void popstring(void);
 
 Modified: head/bin/sh/main.c
 ==============================================================================
 --- head/bin/sh/main.c	Sat Jun 13 21:10:41 2009	(r194127)
 +++ head/bin/sh/main.c	Sat Jun 13 21:17:45 2009	(r194128)
 @@ -178,7 +178,7 @@ state2:
  state3:
  	state = 4;
  	if (minusc) {
 -		evalstring(minusc, 0);
 +		evalstring(minusc, sflag ? 0 : EV_EXIT);
  	}
  	if (sflag || minusc == NULL) {
  state4:	/* XXX ??? - why isn't this before the "if" statement */
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: jilles 
State-Changed-When: Sat Jun 13 21:46:37 UTC 2009 
State-Changed-Why:  
Fixed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=113860 
>Unformatted:
