From jlh@felucia.tataz.chchile.org  Sun Feb 20 16:50:40 2011
Return-Path: <jlh@felucia.tataz.chchile.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 487FD106566C
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 20 Feb 2011 16:50:40 +0000 (UTC)
	(envelope-from jlh@felucia.tataz.chchile.org)
Received: from smtpfb1-g21.free.fr (smtpfb1-g21.free.fr [212.27.42.9])
	by mx1.freebsd.org (Postfix) with ESMTP id A29158FC14
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 20 Feb 2011 16:50:37 +0000 (UTC)
Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5])
	by smtpfb1-g21.free.fr (Postfix) with ESMTP id 9B98F2E22B
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 20 Feb 2011 17:32:31 +0100 (CET)
Received: from endor.tataz.chchile.org (unknown [82.233.239.98])
	by smtp5-g21.free.fr (Postfix) with ESMTP id 46ED0D481B8;
	Sun, 20 Feb 2011 17:32:24 +0100 (CET)
Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9])
	by endor.tataz.chchile.org (Postfix) with ESMTP id E9A8D33CF9;
	Sun, 20 Feb 2011 16:32:22 +0000 (UTC)
Received: by felucia.tataz.chchile.org (Postfix, from userid 1000)
	id CAEBBA1498; Sun, 20 Feb 2011 16:32:22 +0000 (UTC)
Message-Id: <20110220163222.CAEBBA1498@felucia.tataz.chchile.org>
Date: Sun, 20 Feb 2011 16:32:22 +0000 (UTC)
From: Jeremie Le Hen <jeremie@le-hen.org>
Reply-To: Jeremie Le Hen <jeremie@le-hen.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc: jeremie@le-hen.org
Subject: [patch] Force stdio output streams to line-buffered mode
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         154915
>Category:       kern
>Synopsis:       [libc] [patch] Force stdio output streams to line-buffered mode
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 20 17:00:20 UTC 2011
>Closed-Date:    Thu Apr 05 08:56:09 UTC 2012
>Last-Modified:  Thu Apr 05 08:56:09 UTC 2012
>Originator:     Jeremie Le Hen
>Release:        FreeBSD 8.1-STABLE i386
>Organization:
>Environment:
System: FreeBSD felucia.tataz.chchile.org

>Description:
	I've been annoyed multiple time when running a command such like
	    iostat -x 1 | cat -n | grep -v ad10

	The problem stems from two factors:
	  - cat's stdio sees that its stdout is not a terminal, so stdout is
	    full buffered and not line-buffered;
	  - iostat produces output too slowly so the aforementioned buffer takes
	    numerous seconds to be filled and flushed to the last command.

	This problems is not specific to FreeBSD, it is actually a consequence
	of POSIX specification.  I've checked this on Solaris and Linux.

	I've attached a small patch for stdio, so if the environment variable
	STDIO_IOLBF is set, the output streams will be line-oriented by default.
	    iostat -x 1 | env STDIO_IOLBF=1 cat -n | grep -v ad10

>How-To-Repeat:

>Fix:

--- STDIO_IOLBF.diff begins here ---
diff -rup /usr/src.orig/lib/libc/stdio/makebuf.c /usr/src/lib/libc/stdio/makebuf.c
--- /usr/src.orig/lib/libc/stdio/makebuf.c	2009-08-03 10:13:06.000000000 +0200
+++ /usr/src/lib/libc/stdio/makebuf.c	2011-02-19 19:09:56.000000000 +0100
@@ -59,6 +59,7 @@ __smakebuf(fp)
 	FILE *fp;
 {
 	void *p;
+	char *bmode;
 	int flags;
 	size_t size;
 	int couldbetty;
@@ -79,7 +80,8 @@ __smakebuf(fp)
 	flags |= __SMBF;
 	fp->_bf._base = fp->_p = p;
 	fp->_bf._size = size;
-	if (couldbetty && isatty(fp->_file))
+	if (((bmode = getenv("STDIO_IOLBF")) && bmode[0] != '\0') ||
+	    (couldbetty && isatty(fp->_file)))
 		flags |= __SLBF;
 	fp->_flags |= flags;
 }
diff -rup /usr/src.orig/lib/libc/stdio/setbuf.3 /usr/src/lib/libc/stdio/setbuf.3
--- /usr/src.orig/lib/libc/stdio/setbuf.3	2009-08-03 10:13:06.000000000 +0200
+++ /usr/src/lib/libc/stdio/setbuf.3	2011-02-19 19:09:13.000000000 +0100
@@ -79,7 +79,9 @@ and an optimally-sized buffer is obtaine
 If a stream refers to a terminal
 (as
 .Dv stdout
-normally does) it is line buffered.
+normally does), or the environment variable
+.Ev STDIO_IOLBF
+is set, it is line buffered.
 The standard error stream
 .Dv stderr
 is always unbuffered.
@@ -176,6 +178,12 @@ The
 function returns what the equivalent
 .Fn setvbuf
 would have returned.
+.Sh ENVIRONMENT
+.Bl -tag -width ".Ev STDIO_IOLBF"
+If the environment variable
+.Ev STDIO_IOLBF
+is set, output streams will be line-buffered by default
+even when not referring to a terminal.
 .Sh SEE ALSO
 .Xr fclose 3 ,
 .Xr fopen 3 ,
diff -rup /usr/src.orig/lib/libc/stdio/stdio.3 /usr/src/lib/libc/stdio/stdio.3
--- /usr/src.orig/lib/libc/stdio/stdio.3	2009-08-03 10:13:06.000000000 +0200
+++ /usr/src/lib/libc/stdio/stdio.3	2011-02-19 12:56:00.000000000 +0100
@@ -137,7 +137,8 @@ an interactive or
 .Dq terminal
 device, as determined by the
 .Xr isatty 3
-function.
+function (this can be overriden with an environment variable, see
+.Xr setvbuf 3  ) .
 In fact,
 .Em all
 freshly-opened streams that refer to terminal devices
--- STDIO_IOLBF.diff ends here ---


>Release-Note:
>Audit-Trail:

From: Oliver Fromme <olli@lurza.secnetix.de>
To: bug-followup@FreeBSD.org, jeremie@le-hen.org
Cc:  
Subject: Re: kern/154915: [libc] [patch] Force stdio output streams to line-buffered mode
Date: Thu, 16 Feb 2012 14:49:50 +0100 (CET)

 I think introducing an environment variable for this purpose
 is a bad hack.  I would advise against this.
 
 Many tools already have options for unbuffered or line-buffered
 output (for example cat -u), and there are also other ways to
 circumvent such problems.
 
 For example, the problem quoted in the PR can be solved like this,
 using the -u option of cat:
 
     $ iostat -x 1 | cat -un | grep ad1
 
 or avoiding cat completely (also might be more efficient,
 saving one process and one pipe, though I haven't benchmarked
 this):
 
     $ iostat -x 1 | awk '{n+=1} /ad1/{print n, $0}'
 
 For certain other cases, I have the following alias in my ~/.zshrc
 that simulates a TTY environment for a tool so it is forced to
 use line-buffered output:
 
     alias intty='script -qt0 /dev/null'
 
 So I can write:
 
     $ intty sometool -args | grep ...
 
 However, the intty alias only works when it is the first command
 in a pipeline (this is a limitation of the "script" command).
 In the above example, the cat command is not the first, but a
 subshell can be used to work around this:
 
     $ intty sh -c 'iostat -x 1 | cat -n' | grep ad1
 
 (It should work with any shell that supports aliases, not just
 zsh, of course.)
 
 I suggest closing this PR.
 
 Best regards
 Oliver

From: Jeremie Le Hen <jeremie@le-hen.org>
To: Oliver Fromme <olli@lurza.secnetix.de>
Cc: bug-followup@FreeBSD.org, jeremie@le-hen.org
Subject: Re: kern/154915: [libc] [patch] Force stdio output streams to
 line-buffered mode
Date: Fri, 17 Feb 2012 18:53:06 +0100

 --n8g4imXOkfNTN/H1
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 Hi Oliver,
 
 On Thu, Feb 16, 2012 at 02:49:50PM +0100, Oliver Fromme wrote:
 > I think introducing an environment variable for this purpose
 > is a bad hack.  I would advise against this.
 > 
 > Many tools already have options for unbuffered or line-buffered
 > output (for example cat -u), and there are also other ways to
 > circumvent such problems.
 > 
 > For example, the problem quoted in the PR can be solved like this,
 > using the -u option of cat:
 > 
 >     $ iostat -x 1 | cat -un | grep ad1
 
 This doesn't help if you enclose a command that doesn't have such an
 option between pipes.  For instance:
 
 $ iostat -x 1 | cat -u | tr MG mg | cat -u | grep -n ad1
 
 > or avoiding cat completely (also might be more efficient,
 > saving one process and one pipe, though I haven't benchmarked
 > this):
 > 
 >     $ iostat -x 1 | awk '{n+=1} /ad1/{print n, $0}'
 
 Right we can almost always replace a pipeline of commands with an awk
 (or whatever) script.  But besides being sometimes inconvenient, saving
 a process or pipe does not always leads to performance benefits.  The
 very example you took is a good illustration of my point: I sometimes
 have to dig into big log files and awk is naturally very powerful.  Yet
 I noticed a *huge* performance gap between (gnu) grep and awk:
 
 $ time grep -n 'include.*stdio' 'FreeNode/##c.log' > /dev/null
 	1.10 real         0.97 user         0.12 sys
 $ time awk '{n+=1} /include.*stdio/{print n, $0}' 'FreeNode/##c.log' > /dev/null
 	3.74 real         3.63 user         0.08 sys
     
 > For certain other cases, I have the following alias in my ~/.zshrc
 > that simulates a TTY environment for a tool so it is forced to
 > use line-buffered output:
 > 
 >     alias intty='script -qt0 /dev/null'
 > 
 > So I can write:
 > 
 >     $ intty sometool -args | grep ...
 > 
 > However, the intty alias only works when it is the first command
 > in a pipeline (this is a limitation of the "script" command).
 
 I wasn't aware of that.  Out of curiosity, I tried the following
 command in order to test this:
 
 $ vmstat  1 | script -qt 0 /dev/null grep . | cat
 
 And each line in output is printed twice.  I didn't have time to
 investigate though.
 
 > In the above example, the cat command is not the first, but a
 > subshell can be used to work around this:
 > 
 >     $ intty sh -c 'iostat -x 1 | cat -n' | grep ad1
 > 
 > (It should work with any shell that supports aliases, not just
 > zsh, of course.)
 > 
 > I suggest closing this PR.
 
 Well, an evil hack VS an even eviler workaround which is not as powerful
 IMHO ;).
 
 In fact, I made a far more better of version of my patch which doesn't
 modify libc.  Instead it introduces a new shared library (libstdbuf.so)
 which is still controlled by environment variables, and an helper
 command (stdbuf(1)) is provided to do the environment-fu (including
 LD_PRELOAD) in your stead.
 
 Unfortuntately I forgot to update it in the PR database.  You will find
 it attached to this email, I think the bug tracker will show it in the
 PR afterward.
 
 Can you tell me if it better fits your expectations in matter of
 "low-hackiness"?  The standard code path is not modified at all.
 
 I would be glad if we could proceed with this, this is a feature I often
 miss.
 
 Thanks a lot for taking time to answer.
 Regards,
 -- 
 Jeremie Le Hen
 
 Men are born free and equal.  Later on, they're on their own.
 				Jean Yanne
 
 --n8g4imXOkfNTN/H1
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="stdbuf.diff"
 
 diff -urNp src.HEAD_20111506/lib/libc/stdio/setbuf.3 src/lib/libc/stdio/setbuf.3
 --- src.HEAD_20111506/lib/libc/stdio/setbuf.3	2007-01-09 01:28:07.000000000 +0100
 +++ src/lib/libc/stdio/setbuf.3	2011-08-04 19:00:49.000000000 +0200
 @@ -30,7 +30,7 @@
  .\" SUCH DAMAGE.
  .\"
  .\"     @(#)setbuf.3	8.1 (Berkeley) 6/4/93
 -.\" $FreeBSD: src/lib/libc/stdio/setbuf.3,v 1.17 2007/01/09 00:28:07 imp Exp $
 +.\" $FreeBSD: src/lib/libc/stdio/setbuf.3,v 1.17.10.1 2009/08/03 08:13:06 kensmith Exp $
  .\"
  .Dd June 4, 1993
  .Dt SETBUF 3
 @@ -83,6 +83,9 @@ normally does) it is line buffered.
  The standard error stream
  .Dv stderr
  is always unbuffered.
 +Note that these defaults maybe be altered using the
 +.Xr stdbuf 1
 +utility.
  .Pp
  The
  .Fn setvbuf
 @@ -177,6 +180,7 @@ function returns what the equivalent
  .Fn setvbuf
  would have returned.
  .Sh SEE ALSO
 +.Xr stdbuf 1 ,
  .Xr fclose 3 ,
  .Xr fopen 3 ,
  .Xr fread 3 ,
 diff -urNp src.HEAD_20111506/lib/libstdbuf/Makefile src/lib/libstdbuf/Makefile
 --- src.HEAD_20111506/lib/libstdbuf/Makefile	1970-01-01 01:00:00.000000000 +0100
 +++ src/lib/libstdbuf/Makefile	2011-08-04 18:49:59.000000000 +0200
 @@ -0,0 +1,15 @@
 +# $FreeBSD: src/lib/libftpio/Makefile,v 1.17.2.1 2009/08/03 08:13:06 kensmith Exp $
 +
 +.include <bsd.own.mk>
 +
 +LIB=		stdbuf
 +SRCS=		stdbuf.c
 +SHLIB_MAJOR=	1
 +MAN=		libstdbuf.3
 +
 +CFLAGS+=	-I${.CURDIR}
 +LDADD=		-lutil
 +
 +WARNS?=		6
 +
 +.include <bsd.lib.mk>
 diff -urNp src.HEAD_20111506/lib/libstdbuf/libstdbuf.3 src/lib/libstdbuf/libstdbuf.3
 --- src.HEAD_20111506/lib/libstdbuf/libstdbuf.3	1970-01-01 01:00:00.000000000 +0100
 +++ src/lib/libstdbuf/libstdbuf.3	2011-08-04 18:47:04.000000000 +0200
 @@ -0,0 +1,111 @@
 +.\" Copyright (c) 2011 Jeremie Le Hen
 +.\" All rights reserved.
 +.\"
 +.\" Redistribution and use in source and binary forms, with or without
 +.\" modification, are permitted provided that the following conditions
 +.\" are met:
 +.\" 1. Redistributions of source code must retain the above copyright
 +.\"    notice, this list of conditions and the following disclaimer.
 +.\" 2. Redistributions in binary form must reproduce the above copyright
 +.\"    notice, this list of conditions and the following disclaimer in the
 +.\"    documentation and/or other materials provided with the distribution.
 +.\"
 +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
 +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 +.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 +.\" SUCH DAMAGE.
 +.\"
 +.\" $FreeBSD$
 +.\"
 +.Dd August 4, 2011
 +.Dt LIBSTDBUF 3
 +.Os
 +.Sh NAME
 +.Nm libstdbuf
 +.Nd preloaded library to change standard streams initial buffering
 +.Sh DESCRIPTION
 +The
 +.Nm
 +library is meant to be preloaded with the
 +.Ev LD_PRELOAD
 +environment variable to as to change the initial buffering
 +of standard input, standard output and standard error streams.
 +.Pp
 +Although you may load and configure this library manually,
 +an utility,
 +.Xr stdbuf 1 ,
 +can be used to run a command with the appropriate environment variables.
 +.Sh ENVIRONMENT
 +Each stream can be configured indepentently through the following
 +environment variables (values are defined below):
 +.Bl -tag -width size -offset indent
 +.It Ev STDBUF_0
 +Initial buffering definition for the standard input stream
 +.It Ev STDBUF_1
 +Initial buffering definition for the standard output stream
 +.It Ev STDBUF_2
 +Initial buffering definition for the standard error stream
 +.It Ev _STDBUF_I
 +GNU-compatible variable for
 +.Ev STDBUF_0
 +.It Ev _STDBUF_O
 +GNU-compatible variable equivalent to
 +.Ev STDBUF_1
 +.It Ev _STDBUF_E
 +GNU-compatible variable equivalent to
 +.Ev STDBUF_2
 +.El
 +.Pp
 +Each variable may take one of the following values:
 +.Bl -tag -width size -offset indent
 +.It Qq 0
 +unbuffered
 +.It Qq L
 +line buffered
 +.It Qq B
 +fully buffered with the default buffer size
 +.It Ar size
 +fully buffered with a buffer of
 +.Ar size
 +bytes
 +.El
 +.Sh EXAMPLE
 +In the following example, the stdout stream of the
 +.Xr awk 1
 +command
 +will be fully buffered by default because it does not refer
 +to a terminal.
 +.Nm
 +is used to force it to be line-buffered so
 +.Xr vmstat 8 Ns 's
 +output will not stall until the full buffer fills.
 +.Bd -literal -offset indent
 +# vmstat 1 | LD_PRELOAD=/usr/lib/libstdbuf.so \\
 +    STDBUF_1=L awk '$2 > 1 || $3 > 1' | cat -n
 +.Ed
 +.Pp
 +See also the manpage of
 +.Xr stdbuf 1
 +for a simpler way to do this.
 +.Sh HISTORY
 +The
 +.Nm
 +library first appeared in
 +.Fx 9.0 .
 +.Sh AUTHORS
 +.An -nosplit
 +The original idea of the
 +.Nm
 +command comes from
 +.An Padraig Brady
 +who implemented it in the GNU coreutils.
 +.An Jeremie Le Hen
 +implemented it on
 +.Fx .
 diff -urNp src.HEAD_20111506/lib/libstdbuf/stdbuf.c src/lib/libstdbuf/stdbuf.c
 --- src.HEAD_20111506/lib/libstdbuf/stdbuf.c	1970-01-01 01:00:00.000000000 +0100
 +++ src/lib/libstdbuf/stdbuf.c	2011-08-04 18:17:07.000000000 +0200
 @@ -0,0 +1,105 @@
 +/*-
 + * Copyright (c) 2011 Jeremie Le Hen
 + * All rights reserved.
 + *
 + * Redistribution and use in source and binary forms, with or without
 + * modification, are permitted provided that the following conditions
 + * are met:
 + * 1. Redistributions of source code must retain the above copyright
 + *    notice, this list of conditions and the following disclaimer.
 + * 2. Redistributions in binary form must reproduce the above copyright
 + *    notice, this list of conditions and the following disclaimer in the
 + *    documentation and/or other materials provided with the distribution.
 + *
 + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
 + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 + * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 + * SUCH DAMAGE.
 + *
 + * $FreeBSD$
 + */
 +
 +#include <err.h>
 +#include <errno.h>
 +#include <limits.h>
 +#include <stdio.h>
 +#include <stdlib.h>
 +#include <string.h>
 +#include <libutil.h>
 +
 +static const char *
 +stream_name(FILE *s)
 +{
 +	if (s == stdin)
 +		return "stdin";
 +	if (s == stdout)
 +		return "stdout";
 +	if (s == stderr)
 +		return "stderr";
 +	/* This should not happen. */
 +	abort();
 +}
 +
 +static void
 +change_buf(FILE *s, const char *bufmode)
 +{
 +	size_t bufsiz;
 +	uint64_t sizearg;
 +	int mode;
 +
 +	bufsiz = 0;
 +	if (bufmode[0] == '0' && bufmode[1] == '\0')
 +		mode = _IONBF;
 +	else if (bufmode[0] == 'L' && bufmode[1] == '\0')
 +		mode = _IOLBF;
 +	else if (bufmode[0] == 'B' && bufmode[1] == '\0') {
 +		mode = _IOFBF;
 +		bufsiz = 0;
 +	} else {
 +		errno = 0;
 +		if (expand_number(bufmode, &sizearg) == -1) {
 +			warn("Wrong buffer mode '%s' for %s", bufmode,
 +			    stream_name(s));
 +			return;
 +		}
 +		if (sizearg > SIZE_T_MAX) {
 +			warn("Buffer size too big for %s", stream_name(s));
 +			return;
 +		}
 +		mode = _IOFBF;
 +		bufsiz = (size_t)sizearg;
 +	}
 +	if (setvbuf(s, NULL, mode, bufsiz) != 0)
 +		warn("Cannot set buffer mode '%s' for %s", bufmode,
 +		    stream_name(s));
 +}
 +
 +__attribute__ ((constructor)) static void
 +stdbuf(void)
 +{
 +	char *i_mode, *o_mode, *e_mode;
 +
 +	i_mode = getenv("STDBUF_0");
 +	if (i_mode == NULL)
 +		i_mode = getenv("_STDBUF_I");
 +	o_mode = getenv("STDBUF_1");
 +	if (o_mode == NULL)
 +		o_mode = getenv("_STDBUF_O");
 +	e_mode = getenv("STDBUF_2");
 +	if (e_mode == NULL)
 +		e_mode = getenv("_STDBUF_E");
 +
 +	if (e_mode)
 +		change_buf(stderr, e_mode);
 +	if (i_mode)
 +		change_buf(stdin, i_mode);
 +	if (o_mode)
 +		change_buf(stdout, o_mode);
 +}
 diff -urNp src.HEAD_20111506/usr.bin/stdbuf/Makefile src/usr.bin/stdbuf/Makefile
 --- src.HEAD_20111506/usr.bin/stdbuf/Makefile	1970-01-01 01:00:00.000000000 +0100
 +++ src/usr.bin/stdbuf/Makefile	2011-08-02 19:16:30.000000000 +0200
 @@ -0,0 +1,8 @@
 +# $Id$
 +
 +PROG=   stdbuf
 +SRCS=   stdbuf.c
 +
 +WARNS?=	6
 +
 +.include <bsd.prog.mk>
 Files src.HEAD_20111506/usr.bin/stdbuf/sh.core and src/usr.bin/stdbuf/sh.core differ
 diff -urNp src.HEAD_20111506/usr.bin/stdbuf/stdbuf.1 src/usr.bin/stdbuf/stdbuf.1
 --- src.HEAD_20111506/usr.bin/stdbuf/stdbuf.1	1970-01-01 01:00:00.000000000 +0100
 +++ src/usr.bin/stdbuf/stdbuf.1	2011-08-04 18:46:35.000000000 +0200
 @@ -0,0 +1,124 @@
 +.\" Copyright (C) 2011 Jeremie Le Hen
 +.\" All rights reserved.
 +.\"
 +.\" Redistribution and use in source and binary forms, with or without
 +.\" modification, are permitted provided that the following conditions
 +.\" are met:
 +.\" 1. Redistributions of source code and documentation must retain the above
 +.\"    copyright notice, this list of conditions and the following disclaimer.
 +.\" 2. Redistributions in binary form must reproduce the above copyright
 +.\"    notice, this list of conditions and the following disclaimer in the
 +.\"    documentation and/or other materials provided with the distribution.
 +.\" 3. All advertising materials mentioning features or use of this software
 +.\"    must display the following acknowledgement:
 +.\"	This product includes software developed or owned by Caldera
 +.\"	International, Inc.
 +.\" 4. Neither the name of Caldera International, Inc. nor the names of other
 +.\"    contributors may be used to endorse or promote products derived from
 +.\"    this software without specific prior written permission.
 +.\"
 +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA
 +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR
 +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 +.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
 +.\" IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE FOR ANY DIRECT,
 +.\" INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 +.\" (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 +.\" SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
 +.\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
 +.\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 +.\" POSSIBILITY OF SUCH DAMAGE.
 +.\"
 +.\"	$FreeBSD: src/usr.bin/dc/dc.1,v 1.2 2010/01/22 23:50:46 delphij Exp $
 +.\"
 +.Dd August 4, 2011
 +.Dt STDBUF 1
 +.Os
 +.Sh NAME
 +.Nm stdbuf
 +.Nd change standard streams initial buffering
 +.Sh SYNOPSIS
 +.Nm
 +.Op Fl e Ar bufdef
 +.Op Fl i Ar bufdef
 +.Op Fl o Ar bufdef
 +.Op Ar command Op ...
 +.Sh DESCRIPTION
 +.Nm
 +is used to change the initial buffering of standard input,
 +standard output and/or standard error streams for
 +.Ar command .
 +It relies on
 +.Xr libstdbuf 3
 +which is loaded and configured by
 +.Nm
 +through environment variables.
 +.Pp
 +The options are as follows:
 +.Bl -tag -width Ds
 +.It Fl e Ar bufdef
 +Set initial buffering of the standard error stream for
 +.Ar command
 +as defined by
 +.Ar bufdef
 +.Pq see Sx BUFFER DEFINITION .
 +.It Fl i Ar bufdef
 +Set initial buffering of the standard input stream for
 +.Ar command
 +as defined by
 +.Ar bufdef
 +.Pq see Sx BUFFER DEFINITION .
 +.It Fl o Ar bufdef
 +Set initial buffering of the standard output stream for
 +.Ar command
 +as defined by
 +.Ar bufdef
 +.Pq see Sx BUFFER DEFINITION .
 +.El
 +.Sh BUFFER DEFINITION
 +Buffer definition is the same as in
 +.Xr libstdbuf 3 :
 +.Bl -tag -width size -offset indent
 +.It Qq 0
 +unbuffered
 +.It Qq L
 +line buffered
 +.It Qq B
 +fully buffered with the default buffer size
 +.It Ar size
 +fully buffered with a buffer of
 +.Ar size
 +bytes
 +.El
 +.Sh EXAMPLES
 +In the following example, the stdout stream of the
 +.Xr awk 1
 +command
 +will be fully buffered by default because it does not refer
 +to a terminal.
 +.Nm
 +is used to force it to be line-buffered so
 +.Xr vmstat 8 Ns 's
 +output will not stall until the full buffer fills.
 +.Bd -literal -offset indent
 +# vmstat 1 | stdbuf -o L awk '$2 > 1 || $3 > 1' | cat -n
 +.Ed
 +.Sh SEE ALSO
 +.Xr libstdbuf 3 ,
 +.Xr setvbuf 3
 +.Sh HISTORY
 +The
 +.Nm
 +utility first appeared in
 +.Fx 9.0 .
 +.Sh AUTHORS
 +.An -nosplit
 +The original idea of the
 +.Nm
 +command comes from 
 +.An Padraig Brady
 +who implemented it in the GNU coreutils.
 +.An Jeremie Le Hen
 +implemented it on
 +.Fx .
 diff -urNp src.HEAD_20111506/usr.bin/stdbuf/stdbuf.c src/usr.bin/stdbuf/stdbuf.c
 --- src.HEAD_20111506/usr.bin/stdbuf/stdbuf.c	1970-01-01 01:00:00.000000000 +0100
 +++ src/usr.bin/stdbuf/stdbuf.c	2011-08-04 18:17:50.000000000 +0200
 @@ -0,0 +1,107 @@
 +/*-
 + * Copyright (c) 2011 Jeremie Le Hen
 + * All rights reserved.
 + *
 + * Redistribution and use in source and binary forms, with or without
 + * modification, are permitted provided that the following conditions
 + * are met:
 + * 1. Redistributions of source code must retain the above copyright
 + *    notice, this list of conditions and the following disclaimer.
 + * 2. Redistributions in binary form must reproduce the above copyright
 + *    notice, this list of conditions and the following disclaimer in the
 + *    documentation and/or other materials provided with the distribution.
 + *
 + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
 + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 + * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 + * SUCH DAMAGE.
 + *
 + * $FreeBSD$
 + */
 +
 +#include <err.h>
 +#include <stdio.h>
 +#include <stdlib.h>
 +#include <unistd.h>
 +
 +#define	LIBSTDBUF	"/usr/lib/libstdbuf.so"
 +
 +extern char *__progname;
 +
 +static void usage(int);
 +
 +static void
 +usage(int s)
 +{
 +	
 +	fprintf(stderr, "Usage: %s [-e bufdef] [-i bufdef] [-o bufdef] "
 +	    "<command> [args ...]\n", __progname);
 +	fprintf(stderr, "  ``bufdef'' being:\n");
 +	fprintf(stderr, "    - \"0\" to disable buffering;\n");
 +	fprintf(stderr, "    - \"L\" to set line-buffering;\n");
 +	fprintf(stderr, "    - <size> to set full-buffering.\n");
 +	exit(s);
 +}
 +
 +int
 +main(int argc, char *argv[])
 +{
 +	int i;
 +	char *ibuf = NULL, *obuf = NULL, *ebuf = NULL;
 +	char *preload0, *preload1;
 +
 +	while ((i = getopt(argc, argv, ":e:i:o:")) != -1) {
 +		switch (i) {
 +		case 'e':
 +			ebuf = optarg;
 +			break;
 +		case 'i':
 +			ibuf = optarg;
 +			break;
 +		case 'o':
 +			obuf = optarg;
 +			break;
 +		case ':':
 +			warnx("Missing argument for option -%c", optopt);
 +			usage(1);
 +			break;
 +		case '?':
 +		default:
 +			warnx("Unknown option -%c", optopt);
 +			usage(1);
 +			break;
 +		}
 +	}
 +	argc -= optind;
 +	argv += optind;
 +
 +	if (ibuf != NULL && setenv("STDBUF_0", ibuf, 1) == -1)
 +		warn("Failed to set environment variable: %s=%s",
 +		    "STDBUF_0", ibuf);
 +	if (obuf != NULL && setenv("STDBUF_1", obuf, 1) == -1)
 +		warn("Failed to set environment variable: %s=%s",
 +		    "STDBUF_1", obuf);
 +	if (ebuf != NULL && setenv("STDBUF_2", ebuf, 1) == -1)
 +		warn("Failed to set environment variable: %s=%s",
 +		    "STDBUF_2", ebuf);
 +
 +	preload0 = getenv("LD_PRELOAD");
 +	if (preload0 == NULL)
 +		i = asprintf(&preload1, "LD_PRELOAD=" LIBSTDBUF);
 +	else
 +		i = asprintf(&preload1, "LD_PRELOAD=%s:%s", preload0,
 +		    LIBSTDBUF);
 +
 +	if (i < 0 || putenv(preload1) == -1)
 +		warn("Failed to set environment variable: %s", preload1);
 +
 +	execvp(argv[0], argv);
 +	err(2, "%s", argv[0]);
 +}
 
 --n8g4imXOkfNTN/H1--

From: Jeremie Le Hen <jeremie@le-hen.org>
To: Oliver Fromme <olli@lurza.secnetix.de>
Cc: bug-followup@FreeBSD.org, jeremie@le-hen.org
Subject: Re: kern/154915: [libc] [patch] Force stdio output streams to
 line-buffered mode
Date: Thu, 5 Apr 2012 08:55:00 +0200

 Hi Oliver,
 
 I'm top-posting to let you handle this mail quickly.
 
 Can you please close this PR.  The content is messy, the subject is
 misleading and I don't think these points will help the patch to be
 committed.  I will open a new PR with the new patch that doesn't modify
 libc.
 
 Thanks.
 Regards,
 
 
 On Thu, Feb 16, 2012 at 02:49:50PM +0100, Oliver Fromme wrote:
 > I think introducing an environment variable for this purpose
 > is a bad hack.  I would advise against this.
 > 
 > Many tools already have options for unbuffered or line-buffered
 > output (for example cat -u), and there are also other ways to
 > circumvent such problems.
 > 
 > For example, the problem quoted in the PR can be solved like this,
 > using the -u option of cat:
 > 
 >     $ iostat -x 1 | cat -un | grep ad1
 > 
 > or avoiding cat completely (also might be more efficient,
 > saving one process and one pipe, though I haven't benchmarked
 > this):
 > 
 >     $ iostat -x 1 | awk '{n+=1} /ad1/{print n, $0}'
 > 
 > For certain other cases, I have the following alias in my ~/.zshrc
 > that simulates a TTY environment for a tool so it is forced to
 > use line-buffered output:
 > 
 >     alias intty='script -qt0 /dev/null'
 > 
 > So I can write:
 > 
 >     $ intty sometool -args | grep ...
 > 
 > However, the intty alias only works when it is the first command
 > in a pipeline (this is a limitation of the "script" command).
 > In the above example, the cat command is not the first, but a
 > subshell can be used to work around this:
 > 
 >     $ intty sh -c 'iostat -x 1 | cat -n' | grep ad1
 > 
 > (It should work with any shell that supports aliases, not just
 > zsh, of course.)
 > 
 > I suggest closing this PR.
 > 
 > Best regards
 > Oliver
 
 -- 
 Jeremie Le Hen
 
 Men are born free and equal.  Later on, they're on their own.
 				Jean Yanne
State-Changed-From-To: open->closed 
State-Changed-By: linimon 
State-Changed-When: Thu Apr 5 08:55:57 UTC 2012 
State-Changed-Why:  
Closed at submitter's request. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=154915 
>Unformatted:
