From eugen@grosbein.pp.ru  Thu Aug  7 08:14:20 2003
Return-Path: <eugen@grosbein.pp.ru>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id C9C3337B401; Thu,  7 Aug 2003 08:14:20 -0700 (PDT)
Received: from grosbein.pp.ru (D00015.dialonly.kemerovo.su [213.184.66.105])
	by mx1.FreeBSD.org (Postfix) with ESMTP
	id A566A43F93; Thu,  7 Aug 2003 08:14:13 -0700 (PDT)
	(envelope-from eugen@grosbein.pp.ru)
Received: from grosbein.pp.ru (eugen@localhost [127.0.0.1])
	by grosbein.pp.ru (8.12.9/8.12.9) with ESMTP id h77FE5Y7034358;
	Thu, 7 Aug 2003 23:14:05 +0800 (KRAST)
	(envelope-from eugen@grosbein.pp.ru)
Received: (from eugen@localhost)
	by grosbein.pp.ru (8.12.9/8.12.9/Submit) id h77FE4xC034357;
	Thu, 7 Aug 2003 23:14:04 +0800 (KRAST)
Message-Id: <200308071514.h77FE4xC034357@grosbein.pp.ru>
Date: Thu, 7 Aug 2003 23:14:04 +0800 (KRAST)
From: Eugene Grosbein <eugen@grosbein.pp.ru>
Reply-To: Eugene Grosbein <eugen@grosbein.pp.ru>
To: FreeBSD-gnats-submit@freebsd.org
Cc: stable@freebsd.org
Subject: /bin/sh eats memory and CPU infinitely
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         55346
>Category:       bin
>Synopsis:       sh(1) remembers terminated jobs too long
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    jilles
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Aug 07 08:20:05 PDT 2003
>Closed-Date:    Sun May 22 22:45:14 UTC 2011
>Last-Modified:  Sun May 22 22:45:14 UTC 2011
>Originator:     Eugene Grosbein
>Release:        FreeBSD 4.8-STABLE i386
>Organization:
JSC Svyaz-Service
>Environment:
System: FreeBSD grosbein.pp.ru 4.8-STABLE FreeBSD 4.8-STABLE #3: Wed Aug 6 21:50:36 KRAST 2003 eu@grosbein.pp.ru:/usr/local/obj/usr/local/src/sys/DADV i386
	CPUTYPE=i686 and no other optimizations
	
>Description:
	/bin/sh leaks memory and can eat all CPU cycles when
	it become very big.

>How-To-Repeat:

	Test script is 15K in size so it comes gzipped
	and uuencoded. Yes, it is compressed very well.

begin 644 test.sh.gz
M'XL(`"5H,C\"`^W3P0F#`!``L/]-<2+T*WYUF^*!@FBI%=<O=(B"D.R0MNF>
MR]8=<\0U+VOEYWU6CCGMD3]#]L!-Y$-<$!<0%Q`7Q`7$!<0%<0%Q`7%!7$!<
;0%Q`7!`7^&_<8ZUZ91_3OE5\`=@<)[J\.P``
`
end
		
	Run it and watch how /bin/sh slowly increases
	its SIZE, RES, WCPU and CPU values.

	This is an exapmle. My real script grows upto 70Mb in size
	and eats all of idle Pentium-III 866Mhz cycles, it is
	machine-generated and runs 24x7x365. It's twice smaller
	than this test case.
	Now I have to restart it from time to time.

>Fix:

	Unknown for me.

Eugene Grosbein
>Release-Note:
>Audit-Trail:

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Sun, 10 Aug 2003 22:05:35 +0800

 Hi!
 
 I digged this case a little. It seems for me that problem is
 in jobs.c: makejob() increases size of jobtab[] array.
 And I do not see a place where it is decreased.
 
 But I see many loops that are O(njobs).
 
 Eugene Grosbein

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: stable@freebsd.org
Cc: bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Fri, 15 Aug 2003 00:30:35 +0800

 Hi!
 
 It seems /bin/sh in 4.8-STABLE has problem with SIGCHLD processing.
 In short, it often fails to process it correctly, zombies float
 around, jobs are not marked as finished in jobtab[] that fills memory
 and takes much CPU to be processed.
 
 Run this one-liner using /bin/sh and see hundreds of zombies:
 
 #!/bin/sh
 while :; do : & done
 
 A kernel will halt this as soon as it reaches limits and write a message:
 
 /kernel: maxproc limit exceeded by uid 0, please see tuning(7) and login.conf(5).
 
 Eugene Grosbein

From: Doug White <dwhite@gumbysoft.com>
To: Eugene Grosbein <eugen@grosbein.pp.ru>
Cc: stable@freebsd.org, bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Thu, 14 Aug 2003 11:04:13 -0700 (PDT)

 On Fri, 15 Aug 2003, Eugene Grosbein wrote:
 
 > Hi!
 >
 > It seems /bin/sh in 4.8-STABLE has problem with SIGCHLD processing.
 > In short, it often fails to process it correctly, zombies float
 > around, jobs are not marked as finished in jobtab[] that fills memory
 > and takes much CPU to be processed.
 >
 > Run this one-liner using /bin/sh and see hundreds of zombies:
 >
 > #!/bin/sh
 > while :; do : & done
 
 Considering that with this script you are forkbombing your machine as
 root, I think this falls into the "doctor it hurts when I shoot my foot"
 category.
 
 -- 
 Doug White                    |  FreeBSD: The Power to Serve
 dwhite@gumbysoft.com          |  www.FreeBSD.org

From: Eugene Grosbein <eugen@kuzbass.ru>
To: Doug White <dwhite@gumbysoft.com>
Cc: Eugene Grosbein <eugen@grosbein.pp.ru>, stable@freebsd.org,
	bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Fri, 15 Aug 2003 11:14:31 +0800

 Doug White wrote:
 
 > > It seems /bin/sh in 4.8-STABLE has problem with SIGCHLD processing.
 > > In short, it often fails to process it correctly, zombies float
 > > around, jobs are not marked as finished in jobtab[] that fills memory
 > > and takes much CPU to be processed.
 > >
 > > Run this one-liner using /bin/sh and see hundreds of zombies:
 > >
 > > #!/bin/sh
 > > while :; do : & done
 > 
 > Considering that with this script you are forkbombing your machine as
 > root, I think this falls into the "doctor it hurts when I shoot my foot"
 > category.
 
 Perhaps. Anyway, debug shows that jobtab[] in /bin/sh grows indefinitely
 in some scenarios and the reason should be SIGCHLD processing problem.
 
 Eugene

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: stable@freebsd.org
Cc: bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Fri, 15 Aug 2003 21:13:21 +0800

 I think I've found a memory leak in /bin/sh.
 There is a case when dowait() and does frees resources of
 completed job correctly. Here is a patch:
 
 Index: jobs.c
 ===================================================================
 RCS file: /home/ncvs/src/bin/sh/jobs.c,v
 retrieving revision 1.27.2.11
 diff -u -r1.27.2.11 jobs.c
 --- jobs.c	22 Jul 2003 13:11:26 -0000	1.27.2.11
 +++ jobs.c	15 Aug 2003 13:02:23 -0000
 @@ -960,10 +960,8 @@
  				if (jp->state != state) {
  					TRACE(("Job %d: changing state from %d to %d\n", jp - jobtab + 1, jp->state, state));
  					jp->state = state;
 -#if JOBS
  					if (done)
 -						deljob(jp);
 -#endif
 +					    freejob(jp);
  				}
  			}
  		}
  
 Eugene Grosbein

From: Doug White <dwhite@gumbysoft.com>
To: Eugene Grosbein <eugen@grosbein.pp.ru>
Cc: stable@freebsd.org, bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Fri, 15 Aug 2003 19:13:57 -0700 (PDT)

 On Fri, 15 Aug 2003, Eugene Grosbein wrote:
 
 > I think I've found a memory leak in /bin/sh.
 > There is a case when dowait() and does frees resources of
 > completed job correctly. Here is a patch:
 
 Your idea has some merit but your patch does not conform to normal style.
 
 > -#if JOBS
 >  					if (done)
 > -						deljob(jp);
 > -#endif
 > +					    freejob(jp);
 
 You should not remove the ifdef and change the indentation of deljob. Just
 s/del/free and be done with it :-)  I highly doubt anyone needs to disable
 job control in sh, but in the off chance they do, you might as well keep
 the ifdef around.
 
 Note that freejob calls deljob.
 
 -- 
 Doug White                    |  FreeBSD: The Power to Serve
 dwhite@gumbysoft.com          |  www.FreeBSD.org

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: Doug White <dwhite@gumbysoft.com>
Cc: stable@freebsd.org, bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Sun, 17 Aug 2003 00:46:19 +0800

 On Fri, Aug 15, 2003 at 07:13:57PM -0700, Doug White wrote:
 
 > You should not remove the ifdef and change the indentation of deljob. Just
 > s/del/free and be done with it :-)  I highly doubt anyone needs to disable
 > job control in sh, but in the off chance they do, you might as well keep
 > the ifdef around.
 > 
 > Note that freejob calls deljob.
 
 That's why I've removed #ifdef.
 freejob() already has these #ifdefs around deljob().
 However, freejob() is not wrapped with #ifdefs itself. So I've removed
 them here.
 
 Identation is my fault. 
 
 Eugene Grosbein

From: Doug White <dwhite@gumbysoft.com>
To: Eugene Grosbein <eugen@grosbein.pp.ru>
Cc: stable@freebsd.org, bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Sat, 16 Aug 2003 12:00:32 -0700 (PDT)

 On Sun, 17 Aug 2003, Eugene Grosbein wrote:
 
 > On Fri, Aug 15, 2003 at 07:13:57PM -0700, Doug White wrote:
 >
 > > You should not remove the ifdef and change the indentation of deljob. Just
 > > s/del/free and be done with it :-)  I highly doubt anyone needs to disable
 > > job control in sh, but in the off chance they do, you might as well keep
 > > the ifdef around.
 > >
 > > Note that freejob calls deljob.
 >
 > That's why I've removed #ifdef.
 > freejob() already has these #ifdefs around deljob().
 > However, freejob() is not wrapped with #ifdefs itself. So I've removed
 > them here.
 
 Ah, OK, I follow you there. send-pr away.
 
 -- 
 Doug White                    |  FreeBSD: The Power to Serve
 dwhite@gumbysoft.com          |  www.FreeBSD.org

From: Eugene Grosbein <eugen@kuzbass.ru>
To: Doug White <dwhite@gumbysoft.com>
Cc: stable@freebsd.org, bug-followup@freebsd.org
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Sun, 17 Aug 2003 11:50:54 +0800

 Doug White wrote:
 
 > Ah, OK, I follow you there. send-pr away.
 
 It seems that patch replaces memory leak with double-free bug:
 waitforjob() will call freejob() just after dowait().
 
 This corrected version removes extra freejob() call.
 
 --- jobs.c.orig Mon Aug  4 11:48:00 2003
 +++ jobs.c      Sun Aug 17 11:20:55 2003
 @@ -889,8 +889,6 @@
  #endif
         else
                 st = WTERMSIG(status) + 128;
 -       if (! JOBS || jp->state == JOBDONE)
 -               freejob(jp);
         if (int_pending()) {
                 if (WIFSIGNALED(status) && WTERMSIG(status) == SIGINT)
                         kill(getpid(), SIGINT);
 @@ -960,10 +958,8 @@
                                 if (jp->state != state) {
                                         TRACE(("Job %d: changing state from %d
 to %d\n", jp -
 jobtab + 1, jp->state, state));
                                         jp->state = state;
 -#if JOBS
                                         if (done)
 -                                               deljob(jp);
 -#endif
 +                                               freejob(jp);
                                 }
                         }
                 }
 
 
 Eugene Grosbein

From: David Schultz <das@FreeBSD.ORG>
To: Eugene Grosbein <eugen@grosbein.pp.ru>
Cc: stable@FreeBSD.ORG, bug-followup@FreeBSD.ORG
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Sun, 24 Aug 2003 11:56:18 -0700

 On Fri, Aug 15, 2003, Eugene Grosbein wrote:
 > I think I've found a memory leak in /bin/sh.
 > There is a case when dowait() and does frees resources of
 > completed job correctly. Here is a patch:
 > 
 > Index: jobs.c
 > ===================================================================
 > RCS file: /home/ncvs/src/bin/sh/jobs.c,v
 > retrieving revision 1.27.2.11
 > diff -u -r1.27.2.11 jobs.c
 > --- jobs.c	22 Jul 2003 13:11:26 -0000	1.27.2.11
 > +++ jobs.c	15 Aug 2003 13:02:23 -0000
 > @@ -960,10 +960,8 @@
 >  				if (jp->state != state) {
 >  					TRACE(("Job %d: changing state from %d to %d\n", jp - jobtab + 1, jp->state, state));
 >  					jp->state = state;
 > -#if JOBS
 >  					if (done)
 > -						deljob(jp);
 > -#endif
 > +					    freejob(jp);
 >  				}
 >  			}
 >  		}
 
 I don't think this is right.  This will cause jobs to be freed
 even when they shouldn't be.
 
 The general problem you're complaining about (here and earlier) is
 that /bin/sh only checks for the termination of backgrounded
 children when it displays a prompt, and of course it doesn't do
 that in the middle of a while loop.  I don't know what the various
 standards have to say about this, but the behavior is probably
 just a bug.

From: Eugene Grosbein <eugen@kuzbass.ru>
To: David Schultz <das@FreeBSD.ORG>
Cc: Eugene Grosbein <eugen@grosbein.pp.ru>, stable@FreeBSD.ORG,
	bug-followup@FreeBSD.ORG
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Mon, 25 Aug 2003 20:36:56 +0800

 David Schultz wrote:
 > 
 > On Fri, Aug 15, 2003, Eugene Grosbein wrote:
 > > I think I've found a memory leak in /bin/sh.
 > > There is a case when dowait() and does frees resources of
 > > completed job correctly. Here is a patch:
 > >
 > > Index: jobs.c
 > > ===================================================================
 > > RCS file: /home/ncvs/src/bin/sh/jobs.c,v
 > > retrieving revision 1.27.2.11
 > > diff -u -r1.27.2.11 jobs.c
 > > --- jobs.c    22 Jul 2003 13:11:26 -0000      1.27.2.11
 > > +++ jobs.c    15 Aug 2003 13:02:23 -0000
 > > @@ -960,10 +960,8 @@
 > >                               if (jp->state != state) {
 > >                                       TRACE(("Job %d: changing state from %d to %d\n", jp - jobtab + 1, jp->state, state));
 > >                                       jp->state = state;
 > > -#if JOBS
 > >                                       if (done)
 > > -                                             deljob(jp);
 > > -#endif
 > > +                                         freejob(jp);
 > >                               }
 > >                       }
 > >               }
 > 
 > I don't think this is right.  This will cause jobs to be freed
 > even when they shouldn't be.
 
 Please give me an example when job should not be freed in dowait().
 
 > The general problem you're complaining about (here and earlier) is
 > that /bin/sh only checks for the termination of backgrounded
 > children when it displays a prompt, and of course it doesn't do
 > that in the middle of a while loop.  I don't know what the various
 > standards have to say about this, but the behavior is probably
 > just a bug.
 
 Yes it is. Both bash and zsh do not behave so.

From: Doug White <dwhite@gumbysoft.com>
To: Eugene Grosbein <eugen@kuzbass.ru>
Cc: David Schultz <das@FreeBSD.ORG>, stable@FreeBSD.ORG,
	bug-followup@FreeBSD.ORG
Subject: Re: bin/55346: /bin/sh eats memory and CPU infinitely
Date: Mon, 25 Aug 2003 15:56:12 -0700 (PDT)

 On Mon, 25 Aug 2003, Eugene Grosbein wrote:
 
 > > The general problem you're complaining about (here and earlier) is
 > > that /bin/sh only checks for the termination of backgrounded
 > > children when it displays a prompt, and of course it doesn't do
 > > that in the middle of a while loop.  I don't know what the various
 > > standards have to say about this, but the behavior is probably
 > > just a bug.
 >
 > Yes it is. Both bash and zsh do not behave so.
 
 try 'set -b'.  The man page says its unimplmented, but its worth a spin.
 That or feel free to implement it :)
 
 -- 
 Doug White                    |  FreeBSD: The Power to Serve
 dwhite@gumbysoft.com          |  www.FreeBSD.org

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: bin/55346 : /bin/sh eats memory and CPU infinitely
Date: Sat, 7 Jan 2006 17:15:57 +0700

 This is still the problem for 6.0-RELEASE.
 
 Eugene Grosben

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: bug-followup@freebsd.org
Cc:  
Subject: Re:bin/55346: /bin/sh eats memory and CPU infinitely
Date: Sun, 25 Nov 2007 16:02:41 +0700

 Hi!
 
 The problem is still here for 7.0-BETA2
 
 Eugene Grosbein
Responsible-Changed-From-To: freebsd-bugs->stefanf 
Responsible-Changed-By: kris 
Responsible-Changed-When: Tue Dec 25 14:28:20 UTC 2007 
Responsible-Changed-Why:  
Stefan has worked a lot on sh, so perhaps he will be interested in this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=55346 

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: bin/55346: sh(1) eats memory and CPU infinitely
Date: Sat, 3 Apr 2010 01:48:05 +0700

 Hi!
 
 The problem is still here for 8.0-STABLE.
 
 Eugene Grosbein
Responsible-Changed-From-To: stefanf->jilles 
Responsible-Changed-By: jilles 
Responsible-Changed-When: Sun May 23 22:51:15 UTC 2010 
Responsible-Changed-Why:  
I'm working on this. 

The problem consists of two parts: not checking for termination 
often enough and remembering terminated non-current jobs even 
though $! was not referenced. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=55346 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/55346: commit references a PR
Date: Mon, 24 May 2010 10:36:12 +0000 (UTC)

 Author: jilles
 Date: Mon May 24 10:35:57 2010
 New Revision: 208489
 URL: http://svn.freebsd.org/changeset/base/208489
 
 Log:
   sh: Reap any zombies before forking for a background command.
   
   This prevents accumulating huge amounts of zombies if a script executes
   many background commands but no external commands or subshells.
   
   Note that zombies will not be reaped during long calculations (within
   the shell process) or read builtins, but those actions do not create
   more zombies.
   
   The terminated background commands will also still be remembered by the
   shell.
   
   PR:		bin/55346
 
 Modified:
   head/bin/sh/jobs.c
 
 Modified: head/bin/sh/jobs.c
 ==============================================================================
 --- head/bin/sh/jobs.c	Mon May 24 10:23:49 2010	(r208488)
 +++ head/bin/sh/jobs.c	Mon May 24 10:35:57 2010	(r208489)
 @@ -91,6 +91,7 @@ STATIC void freejob(struct job *);
  STATIC struct job *getjob(char *);
  STATIC pid_t dowait(int, struct job *);
  STATIC pid_t waitproc(int, int *);
 +STATIC void checkzombies(void);
  STATIC void cmdtxt(union node *);
  STATIC void cmdputs(const char *);
  #if JOBS
 @@ -400,7 +401,7 @@ showjobs(int change, int mode)
  	struct job *jp;
  
  	TRACE(("showjobs(%d) called\n", change));
 -	while (dowait(0, (struct job *)NULL) > 0);
 +	checkzombies();
  	for (jobno = 1, jp = jobtab ; jobno <= njobs ; jobno++, jp++) {
  		if (! jp->used)
  			continue;
 @@ -742,6 +743,8 @@ forkshell(struct job *jp, union node *n,
  	TRACE(("forkshell(%%%d, %p, %d) called\n", jp - jobtab, (void *)n,
  	    mode));
  	INTOFF;
 +	if (mode == FORK_BG)
 +		checkzombies();
  	flushall();
  	pid = fork();
  	if (pid == -1) {
 @@ -1056,6 +1059,15 @@ stoppedjobs(void)
  	return (0);
  }
  
 +
 +STATIC void
 +checkzombies(void)
 +{
 +	while (njobs > 0 && dowait(0, NULL) > 0)
 +		;
 +}
 +
 +
  /*
   * Return a string identifying a command (to be printed by the
   * jobs command.
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/55346: commit references a PR
Date: Tue, 29 Jun 2010 22:38:09 +0000 (UTC)

 Author: jilles
 Date: Tue Jun 29 22:37:45 2010
 New Revision: 209600
 URL: http://svn.freebsd.org/changeset/base/209600
 
 Log:
   sh: Forget about terminated background processes sooner.
   
   Unless $! has been referenced for a particular job or $! still contains that
   job's pid, forget about it after it has terminated. If $! has been
   referenced, remember the job until the wait builtin has reported its
   completion (either with the pid as parameter or without parameters).
   
   In interactive mode, jobs are forgotten after termination has been reported,
   which happens before primary prompts and through the jobs builtin. Even
   then, though, remember a job if $! has been referenced.
   
   This is similar to what is suggested by POSIX and should fix most memory
   leaks (which also tend to cause sh to use more CPU time) with long running
   scripts that start background jobs.
   
   Caveats:
   * Repeatedly referencing $! without ever doing 'wait', like
       while :; do foo & echo started foo: $!; sleep 60; done
     will still use a lot of memory and CPU time in the long run.
   * The jobs and jobid builtins do not cause a job to be remembered for longer
     like expanding $! does.
   
   PR:		bin/55346
 
 Modified:
   head/bin/sh/expand.c
   head/bin/sh/jobs.c
   head/bin/sh/jobs.h
   head/bin/sh/sh.1
 
 Modified: head/bin/sh/expand.c
 ==============================================================================
 --- head/bin/sh/expand.c	Tue Jun 29 22:07:53 2010	(r209599)
 +++ head/bin/sh/expand.c	Tue Jun 29 22:37:45 2010	(r209600)
 @@ -818,7 +818,7 @@ varisset(char *name, int nulok)
  {
  
  	if (*name == '!')
 -		return backgndpid != -1;
 +		return backgndpidset();
  	else if (*name == '@' || *name == '*') {
  		if (*shellparam.p == NULL)
  			return 0;
 @@ -891,7 +891,7 @@ varvalue(char *name, int quoted, int sub
  		num = shellparam.nparam;
  		goto numvar;
  	case '!':
 -		num = backgndpid;
 +		num = backgndpidval();
  numvar:
  		expdest = cvtnum(num, expdest);
  		break;
 
 Modified: head/bin/sh/jobs.c
 ==============================================================================
 --- head/bin/sh/jobs.c	Tue Jun 29 22:07:53 2010	(r209599)
 +++ head/bin/sh/jobs.c	Tue Jun 29 22:37:45 2010	(r209600)
 @@ -75,6 +75,7 @@ __FBSDID("$FreeBSD$");
  STATIC struct job *jobtab;	/* array of jobs */
  STATIC int njobs;		/* size of array */
  MKINIT pid_t backgndpid = -1;	/* pid of last background process */
 +MKINIT struct job *bgjob = NULL; /* last background process */
  #if JOBS
  STATIC struct job *jobmru;	/* most recently used job list */
  STATIC pid_t initialpgrp;	/* pgrp of shell on invocation */
 @@ -183,6 +184,7 @@ INCLUDE <stdlib.h>
  
  SHELLPROC {
  	backgndpid = -1;
 +	bgjob = NULL;
  #if JOBS
  	jobctl = 0;
  #endif
 @@ -413,7 +415,11 @@ showjobs(int change, int mode)
  			continue;
  		showjob(jp, 0, mode);
  		jp->changed = 0;
 -		if (jp->state == JOBDONE) {
 +		/* Hack: discard jobs for which $! has not been referenced
 +		 * in interactive mode when they terminate.
 +		 */
 +		if (jp->state == JOBDONE && !jp->remembered &&
 +				(iflag || jp != bgjob)) {
  			freejob(jp);
  		}
  	}
 @@ -431,6 +437,8 @@ freejob(struct job *jp)
  	int i;
  
  	INTOFF;
 +	if (bgjob == jp)
 +		bgjob = NULL;
  	for (i = jp->nprocs, ps = jp->ps ; --i >= 0 ; ps++) {
  		if (ps->cmd != nullstr)
  			ckfree(ps->cmd);
 @@ -477,12 +485,27 @@ waitcmd(int argc, char **argv)
  #endif
  				else
  					retval = WTERMSIG(status) + 128;
 -				if (! iflag)
 +				if (! iflag || ! job->changed)
  					freejob(job);
 +				else {
 +					job->remembered = 0;
 +					if (job == bgjob)
 +						bgjob = NULL;
 +				}
  				in_waitcmd--;
  				return retval;
  			}
  		} else {
 +			for (jp = jobtab ; jp < jobtab + njobs; jp++)
 +				if (jp->used && jp->state == JOBDONE) {
 +					if (! iflag || ! jp->changed)
 +						freejob(jp);
 +					else {
 +						jp->remembered = 0;
 +						if (jp == bgjob)
 +							bgjob = NULL;
 +					}
 +				}
  			for (jp = jobtab ; ; jp++) {
  				if (jp >= jobtab + njobs) {	/* no running procs */
  					in_waitcmd--;
 @@ -623,6 +646,8 @@ makejob(union node *node __unused, int n
  						jp[i].next = &jp[jp[i].next -
  						    jobtab];
  #endif
 +				if (bgjob != NULL)
 +					bgjob = &jp[bgjob - jobtab];
  				/* Relocate `ps' pointers */
  				for (i = 0; i < njobs; i++)
  					if (jp[i].ps == &jobtab[i].ps0)
 @@ -644,6 +669,7 @@ makejob(union node *node __unused, int n
  	jp->changed = 0;
  	jp->nprocs = 0;
  	jp->foreground = 0;
 +	jp->remembered = 0;
  #if JOBS
  	jp->jobctl = jobctl;
  	jp->next = NULL;
 @@ -821,8 +847,13 @@ forkshell(struct job *jp, union node *n,
  			pgrp = jp->ps[0].pid;
  		setpgid(pid, pgrp);
  	}
 -	if (mode == FORK_BG)
 +	if (mode == FORK_BG) {
 +		if (bgjob != NULL && bgjob->state == JOBDONE &&
 +		    !bgjob->remembered && !iflag)
 +			freejob(bgjob);
  		backgndpid = pid;		/* set $! */
 +		bgjob = jp;
 +	}
  	if (jp) {
  		struct procstat *ps = &jp->ps[jp->nprocs++];
  		ps->pid = pid;
 @@ -975,10 +1006,15 @@ dowait(int block, struct job *job)
  				if (jp->state != state) {
  					TRACE(("Job %d: changing state from %d to %d\n", jp - jobtab + 1, jp->state, state));
  					jp->state = state;
 +					if (jp != job) {
 +						if (done && !jp->remembered &&
 +						    !iflag && jp != bgjob)
 +							freejob(jp);
  #if JOBS
 -					if (done)
 -						deljob(jp);
 +						else if (done)
 +							deljob(jp);
  #endif
 +					}
  				}
  			}
  		}
 @@ -1074,6 +1110,21 @@ checkzombies(void)
  }
  
  
 +int
 +backgndpidset(void)
 +{
 +	return backgndpid != -1;
 +}
 +
 +
 +pid_t
 +backgndpidval(void)
 +{
 +	if (bgjob != NULL)
 +		bgjob->remembered = 1;
 +	return backgndpid;
 +}
 +
  /*
   * Return a string identifying a command (to be printed by the
   * jobs command.
 
 Modified: head/bin/sh/jobs.h
 ==============================================================================
 --- head/bin/sh/jobs.h	Tue Jun 29 22:07:53 2010	(r209599)
 +++ head/bin/sh/jobs.h	Tue Jun 29 22:37:45 2010	(r209600)
 @@ -68,6 +68,7 @@ struct job {
  	char used;		/* true if this entry is in used */
  	char changed;		/* true if status has changed */
  	char foreground;	/* true if running in the foreground */
 +	char remembered;	/* true if $! referenced */
  #if JOBS
  	char jobctl;		/* job running under job control */
  	struct job *next;	/* job used after this one */
 @@ -81,7 +82,6 @@ enum {
  	SHOWJOBS_PGIDS		/* PID of the group leader only */
  };
  
 -extern pid_t backgndpid;	/* pid of last background process */
  extern int job_warning;		/* user was warned about stopped jobs */
  extern int in_waitcmd;		/* are we in waitcmd()? */
  extern int in_dowait;		/* are we in dowait()? */
 @@ -98,6 +98,8 @@ struct job *makejob(union node *, int);
  pid_t forkshell(struct job *, union node *, int);
  int waitforjob(struct job *, int *);
  int stoppedjobs(void);
 +int backgndpidset(void);
 +pid_t backgndpidval(void);
  char *commandtext(union node *);
  
  #if ! JOBS
 
 Modified: head/bin/sh/sh.1
 ==============================================================================
 --- head/bin/sh/sh.1	Tue Jun 29 22:07:53 2010	(r209599)
 +++ head/bin/sh/sh.1	Tue Jun 29 22:37:45 2010	(r209600)
 @@ -32,7 +32,7 @@
  .\"	from: @(#)sh.1	8.6 (Berkeley) 5/4/95
  .\" $FreeBSD$
  .\"
 -.Dd May 24, 2010
 +.Dd June 29, 2010
  .Dt SH 1
  .Os
  .Sh NAME
 @@ -1106,6 +1106,10 @@ command executed from the current shell.
  For a
  pipeline, the process ID is that of the last command in the
  pipeline.
 +If this parameter is referenced, the shell will remember
 +the process ID and its exit status until the
 +.Ic wait
 +built-in command reports completion of the process.
  .It Li $0
  (zero) Expands to the name of the shell or shell script.
  .El
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: jilles 
State-Changed-When: Sat Jul 3 20:33:15 UTC 2010 
State-Changed-Why:  
Fixed in 9-CURRENT. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=55346 

From: Niclas Zeising <niclas.zeising@gmail.com>
To: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
Cc: jilles@freebsd.org
Subject: Re: bin/55346: sh(1) remembers terminated jobs too long
Date: Tue, 17 May 2011 15:53:19 +0200

 Hi!
 This has been sitting in current for almost a year, but has not been
 MFCd. If it's MFCable, I believe it is time to do so. :)
 Regards!
 -- 
 Niclas

From: Jilles Tjoelker <jilles@stack.nl>
To: Niclas Zeising <niclas.zeising@gmail.com>
Cc: bug-followup@FreeBSD.org, eugen@grosbein.pp.ru
Subject: Re: bin/55346: sh(1) remembers terminated jobs too long
Date: Fri, 20 May 2011 14:58:12 +0200

 On Tue, May 17, 2011 at 03:53:19PM +0200, Niclas Zeising wrote:
 > > [forget about terminated jobs if $! has not been referenced]
 > This has been sitting in current for almost a year, but has not been
 > MFCd. If it's MFCable, I believe it is time to do so. :)
 
 I am fairly paranoid about MFCing sh(1) changes. This change seems
 fairly safe, but may still break things if a script obtains a pid using
 $! in a subprocess or in a way not involving $! such as 'jobid' or
 'jobs' and then expects 'wait' to return the correct exit status for the
 process.
 
 -- 
 Jilles Tjoelker

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: bin/55346: commit references a PR
Date: Sun, 22 May 2011 22:28:26 +0000 (UTC)

 Author: jilles
 Date: Sun May 22 22:28:07 2011
 New Revision: 222208
 URL: http://svn.freebsd.org/changeset/base/222208
 
 Log:
   MFC r208489,r216208: sh: Reap any zombies before forking for a background
   command.
   
   This prevents accumulating huge amounts of zombies if a script executes
   many background commands but no external commands or subshells.
   
   Note that zombies will not be reaped during long calculations (within
   the shell process) or read builtins, but those actions do not create
   more zombies.
   
   The terminated background commands will also still be remembered by the
   shell.
   
   r216208 fixes a bug in r208489 that could cause a multi-command pipeline to
   be marked as done before all processes had been created.
   
   PR:		bin/55346
 
 Modified:
   stable/8/bin/sh/jobs.c
 Directory Properties:
   stable/8/bin/sh/   (props changed)
 
 Modified: stable/8/bin/sh/jobs.c
 ==============================================================================
 --- stable/8/bin/sh/jobs.c	Sun May 22 22:17:06 2011	(r222207)
 +++ stable/8/bin/sh/jobs.c	Sun May 22 22:28:07 2011	(r222208)
 @@ -91,6 +91,7 @@ static void freejob(struct job *);
  static struct job *getjob(char *);
  static pid_t dowait(int, struct job *);
  static pid_t waitproc(int, int *);
 +static void checkzombies(void);
  static void cmdtxt(union node *);
  static void cmdputs(const char *);
  #if JOBS
 @@ -400,7 +401,7 @@ showjobs(int change, int mode)
  	struct job *jp;
  
  	TRACE(("showjobs(%d) called\n", change));
 -	while (dowait(0, (struct job *)NULL) > 0);
 +	checkzombies();
  	for (jobno = 1, jp = jobtab ; jobno <= njobs ; jobno++, jp++) {
  		if (! jp->used)
  			continue;
 @@ -742,6 +743,8 @@ forkshell(struct job *jp, union node *n,
  	TRACE(("forkshell(%%%td, %p, %d) called\n", jp - jobtab, (void *)n,
  	    mode));
  	INTOFF;
 +	if (mode == FORK_BG && (jp == NULL || jp->nprocs == 0))
 +		checkzombies();
  	flushall();
  	pid = fork();
  	if (pid == -1) {
 @@ -948,7 +951,7 @@ dowait(int block, struct job *job)
  	INTOFF;
  	thisjob = NULL;
  	for (jp = jobtab ; jp < jobtab + njobs ; jp++) {
 -		if (jp->used) {
 +		if (jp->used && jp->nprocs > 0) {
  			done = 1;
  			stopped = 1;
  			for (sp = jp->ps ; sp < jp->ps + jp->nprocs ; sp++) {
 @@ -1062,6 +1065,15 @@ stoppedjobs(void)
  	return (0);
  }
  
 +
 +static void
 +checkzombies(void)
 +{
 +	while (njobs > 0 && dowait(0, NULL) > 0)
 +		;
 +}
 +
 +
  /*
   * Return a string identifying a command (to be printed by the
   * jobs command.
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: jilles 
State-Changed-When: Sun May 22 22:40:40 UTC 2011 
State-Changed-Why:  
This is fixed in 9-current and the part of the fix that does not change 
the language has been merged to 8-stable. 

Forgetting jobs sooner if $! has not been referenced changes the language, 
possibly breaking old scripts, and I do not like such a change in a -stable 
branch. As a workaround you can try inserting the command 
jobs >/dev/null 
which will cause the shell to forget about terminated jobs (in 9-current 
that only applies to jobs for which $! has not been referenced). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=55346 
>Unformatted:
