From nobody@FreeBSD.org  Mon Jul  1 17:34:03 2002
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id AF57437B400
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  1 Jul 2002 17:34:03 -0700 (PDT)
Received: from www.freebsd.org (www.FreeBSD.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 74F5843E09
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  1 Jul 2002 17:34:03 -0700 (PDT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.4/8.12.4) with ESMTP id g620Y3OT080656
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 1 Jul 2002 17:34:03 -0700 (PDT)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.4/8.12.4/Submit) id g620Y3As080655;
	Mon, 1 Jul 2002 17:34:03 -0700 (PDT)
Message-Id: <200207020034.g620Y3As080655@www.freebsd.org>
Date: Mon, 1 Jul 2002 17:34:03 -0700 (PDT)
From: Rob Anderson <rob@isilon.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: sed buffers one line of output before writing to stdout
X-Send-Pr-Version: www-1.0

>Number:         40101
>Category:       bin
>Synopsis:       sed buffers one line of output before writing to stdout
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    tjr
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jul 01 17:40:01 PDT 2002
>Closed-Date:    Wed Jul 17 02:39:16 PDT 2002
>Last-Modified:  Wed Jul 17 02:39:16 PDT 2002
>Originator:     Rob Anderson
>Release:        4.5-STABLE
>Organization:
Isilon Systems Inc.
>Environment:
FreeBSD beasly.isilon.com 4.5-STABLE FreeBSD 4.5-STABLE #3: Thu Apr 18 15:58:15 PDT 2002 toor@beasly.endurantsystems-internal.com:/usr/src/sys/compile/ROB  i386
>Description:
sed buffers a full line before writing to stdout. For example, if I enter "sed 's/foo/bar/'" at the prompt, I get the following interaction:

blah   <--- I enter this, get no response
foobar <--- I enter this...
blah   <--- suddenly, the response to my 'blah' is printed
stuff  <--- I enter this...
barbar <--- now, the response to 'foobar' is printed

This is _not_ the behavior in Linux sed...Linux sed prints out every line as it is processed. 'perl -pe' also prints out lines as they are processed. Only BSD sed exhibits this property. I couldn't find anything in the manpage that explains why BSD sed behaves this way.
>How-To-Repeat:
See above... repeats every time. Just run sed with any command, and you'll get this behavior.
>Fix:

>Release-Note:
>Audit-Trail:

From: Rob Anderson <rob@isilon.com>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/40101: sed buffers one line of output before writing to stdout
Date: Mon, 1 Jul 2002 17:51:07 -0700 (PDT)

 Also, this strange behavior is not exhibited by 'perl -pe' or awk. It's
 quite specific to BSD sed.
State-Changed-From-To: open->feedback 
State-Changed-By: dougb 
State-Changed-When: Mon Jul 1 19:09:54 PDT 2002 
State-Changed-Why:  

Can you please explain why you think this is a bug? 


Responsible-Changed-From-To: freebsd-bugs->dougb 
Responsible-Changed-By: dougb 
Responsible-Changed-When: Mon Jul 1 19:09:54 PDT 2002 
Responsible-Changed-Why:  

I'm curious. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40101 

From: Rob Anderson <rob@isilon.com>
To: freebsd-gnats-submit@freebsd.org
Cc:  
Subject: Re: bin/40101
Date: Tue, 2 Jul 2002 14:15:18 -0700 (PDT)

 I think it's a bug because:
 
 - The behavior is non-standard with respect to other similar BSD
 filtering tools, Linux sed, and almost certainly other UNIX sed's.
 - The behavior isn't useful; the user will generally want the output
 back immediately. At the very least, the user will want an option to
 _not_ buffer output

From: Rob Anderson <rob@isilon.com>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/40101: sed buffers one line of output before writing to stdout
Date: Tue, 2 Jul 2002 14:22:48 -0700 (PDT)

 I think it's a bug because:
 
 - The behavior is non-standard with respect to other similar BSD
 filtering tools, Linux sed, and almost certainly other UNIX sed's.
 - The behavior isn't useful; the user will generally want the output
 back immediately. At the very least, the user will want an option to
 _not_ buffer output

From: Tim Robbins <tjr@FreeBSD.ORG>
To: bug-followup@FreeBSD.ORG
Cc: Rob Anderson <rob@isilon.com>
Subject: Re: bin/40101: sed buffers one line of output before writing to stdout
Date: Wed, 3 Jul 2002 21:48:24 +1000

 This patch makes sed use a function, lastline(), instead of a global variable
 to detect whether the current line is the last one of the file. The function
 only checks whether the last line of input has been reached when it's
 requested, instead of checking after each line is read. This is more or
 less the way GNU sed does it.
 
 Most of the changes are related to mf_fgets()'s "f" variable being given file
 scope, and being renamed to "curfile" because "f" is too short a name for a
 variable with large scope.
 
 It's not always possible to avoid the one line delay effect:
 sed -n -e '$p' -e '$!p'
 
 (FWIW, I disagree with your calling this "non-standard". I cannot find any
 sed implementations other than GNU sed which behave this way. That said,
 this seems to be a more logical behaviour than the current one.)
 
 
 Index: extern.h
 ===================================================================
 RCS file: /home/ncvs/src/usr.bin/sed/extern.h,v
 retrieving revision 1.9
 diff -u -r1.9 extern.h
 --- extern.h	22 Mar 2002 01:42:27 -0000	1.9
 +++ extern.h	3 Jul 2002 11:29:58 -0000
 @@ -44,7 +44,6 @@
  extern size_t maxnsub;
  extern u_long linenum;
  extern int appendnum;
 -extern int lastline;
  extern int aflag, eflag, nflag;
  extern const char *fname;
  extern int rflags;	/* regex flags to use */
 @@ -54,5 +53,6 @@
  void	 cspace(SPACE *, char *, size_t, enum e_spflag);
  char	*cu_fgets(char *, int, int *);
  int	 mf_fgets(SPACE *, enum e_spflag);
 +int	 lastline(void);
  void	 process(void);
  char	*strregerror(int, regex_t *);
 Index: main.c
 ===================================================================
 RCS file: /home/ncvs/src/usr.bin/sed/main.c,v
 retrieving revision 1.25
 diff -u -r1.25 main.c
 --- main.c	22 Jun 2002 01:42:26 -0000	1.25
 +++ main.c	3 Jul 2002 11:29:58 -0000
 @@ -96,6 +96,8 @@
   */
  static struct s_flist *files, **fl_nextp = &files;
  
 +static FILE *curfile;		/* Current open file */
 +
  int aflag, eflag, nflag;
  int rflags = 0;
  static int rval;		/* Exit status */
 @@ -107,7 +109,6 @@
  const char *fname;		/* File name. */
  const char *inplace;		/* Inplace edit file extension. */
  u_long linenum;
 -int lastline;			/* TRUE on the last line of the last file */
  
  static void add_compunit(enum e_cut, char *);
  static void add_file(char *);
 @@ -298,36 +299,34 @@
  	SPACE *sp;
  	enum e_spflag spflag;
  {
 -	static FILE *f;		/* Current open file */
  	size_t len;
  	char *p;
  	int c;
  	static int firstfile;
  
 -	if (f == NULL) {
 +	if (curfile == NULL) {
  		/* stdin? */
  		if (files->fname == NULL) {
  			if (inplace != NULL)
  				errx(1, "-i may not be used with stdin");
 -			f = stdin;
 +			curfile = stdin;
  			fname = "stdin";
  		}
  		firstfile = 1;
  	}
  
  	for (;;) {
 -		if (f != NULL && (c = getc(f)) != EOF) {
 -			(void)ungetc(c, f);
 +		if (curfile != NULL && (c = getc(curfile)) != EOF) {
 +			(void)ungetc(c, curfile);
  			break;
  		}
  		/* If we are here then either eof or no files are open yet */
 -		if (f == stdin) {
 +		if (curfile == stdin) {
  			sp->len = 0;
 -			lastline = 1;
  			return (0);
  		}
 -		if (f != NULL) {
 -			fclose(f);
 +		if (curfile != NULL) {
 +			fclose(curfile);
  		}
  		if (firstfile == 0) {
  			files = files->next;
 @@ -335,7 +334,6 @@
  			firstfile = 0;
  		if (files == NULL) {
  			sp->len = 0;
 -			lastline = 1;
  			return (0);
  		}
  		if (inplace != NULL) {
 @@ -343,7 +341,7 @@
  				continue;
  		}
  		fname = files->fname;
 -		if ((f = fopen(fname, "r")) == NULL) {
 +		if ((curfile = fopen(fname, "r")) == NULL) {
  			warn("%s", fname);
  			rval = 1;
  			continue;
 @@ -352,28 +350,21 @@
  			unlink(fname);
  	}
  	/*
 -	 * We are here only when f is open and we still have something to
 -	 * read from it.
 +	 * We are here only when curfile is open and we still have something
 +	 * to read from it.
  	 *
  	 * Use fgetln so that we can handle essentially infinite input data.
  	 * Can't use the pointer into the stdio buffer as the process space
  	 * because the ungetc() can cause it to move.
  	 */
 -	p = fgetln(f, &len);
 -	if (ferror(f))
 +	p = fgetln(curfile, &len);
 +	if (ferror(curfile))
  		errx(1, "%s: %s", fname, strerror(errno ? errno : EIO));
  	if (len != 0 && p[len - 1] == '\n')
  		len--;
  	cspace(sp, p, len, spflag);
  
  	linenum++;
 -	if (files->next == NULL) {
 -		if ((c = getc(f)) != EOF) {
 -			(void)ungetc(c, f);
 -		} else {
 -			lastline = 1;
 -		}
 -	}
  
  	return (1);
  }
 @@ -465,4 +456,17 @@
  	freopen(*filename, "w", stdout);
  	*filename = strdup(backup);
  	return 0;
 +}
 +
 +int
 +lastline(void)
 +{
 +	int ch;
 +
 +	if (files->next != NULL)
 +		return (0);
 +	if ((ch = getc(curfile)) == EOF)
 +		return (1);
 +	ungetc(ch, curfile);
 +	return (0);
  }
 Index: process.c
 ===================================================================
 RCS file: /home/ncvs/src/usr.bin/sed/process.c,v
 retrieving revision 1.23
 diff -u -r1.23 process.c
 --- process.c	24 Jun 2002 11:24:02 -0000	1.23
 +++ process.c	3 Jul 2002 11:29:58 -0000
 @@ -268,7 +268,7 @@
   */
  #define	MATCH(a)						\
  	(a)->type == AT_RE ? regexec_e((a)->u.r, ps, 0, 1, psl) :	\
 -	    (a)->type == AT_LINE ? linenum == (a)->u.l : lastline
 +	    (a)->type == AT_LINE ? linenum == (a)->u.l : lastline()
  
  /*
   * Return TRUE if the command applies to the current line.  Sets the inrange
Responsible-Changed-From-To: dougb->tjr 
Responsible-Changed-By: dougb 
Responsible-Changed-When: Wed Jul 3 06:10:13 PDT 2002 
Responsible-Changed-Why:  

Tim has a patch for this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40101 
State-Changed-From-To: feedback->patched 
State-Changed-By: tjr 
State-Changed-When: Wed Jul 3 07:44:22 PDT 2002 
State-Changed-Why:  
Fix committed to -CURRENT. I will MFC it after 2 weeks. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40101 
State-Changed-From-To: patched->closed 
State-Changed-By: tjr 
State-Changed-When: Wed Jul 17 02:38:31 PDT 2002 
State-Changed-Why:  
Change has been MFC'd. Thanks for suggesting this feature. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40101 
>Unformatted:
