From petervd@daemon.vuurwerk.nl  Wed Jul 12 09:46:15 2000
Return-Path: <petervd@daemon.vuurwerk.nl>
Received: from envy.vuurwerk.nl (envy.vuurwerk.nl [194.178.232.112])
	by hub.freebsd.org (Postfix) with SMTP id 0513337B7F5
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 12 Jul 2000 09:46:13 -0700 (PDT)
	(envelope-from petervd@daemon.vuurwerk.nl)
Received: (qmail 67794 invoked from network); 12 Jul 2000 16:46:01 -0000
Received: from kesteren.vuurwerk.nl (HELO daemon.vuurwerk.nl) (194.178.232.59)
  by envy.vuurwerk.nl with SMTP; 12 Jul 2000 16:46:01 -0000
Received: (nullmailer pid 96634 invoked by uid 11109);
	Wed, 12 Jul 2000 16:46:01 -0000
Message-Id: <963420361.560147.96633.nullmailer@daemon.vuurwerk.nl>
Date: Wed, 12 Jul 2000 18:46:01 +0200
From: petervd@vuurwerk.nl
Sender: petervd@daemon.vuurwerk.nl
Reply-To: petervd@vuurwerk.nl
To: FreeBSD-gnats-submit@freebsd.org
Subject: select bug on named pipes
X-Send-Pr-Version: 3.2

>Number:         19871
>Category:       kern
>Synopsis:       select on named pipes always returns 'available for reading'
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    alfred
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jul 12 09:50:00 PDT 2000
>Closed-Date:    Mon Jun 23 12:35:06 PDT 2003
>Last-Modified:  Mon Jun 23 12:35:06 PDT 2003
>Originator:     Peter van Dijk
>Release:        FreeBSD 4.0-STABLE i386
>Organization:
Vuurwerk Internet, Haarlem, The Netherlands
>Environment:

Standard FreeBSD 4.0-STABLE kernel/world (last cvsup+new world/kernel around
3 months ago).

>Description:

When select()'ing on a named pipe for reading, it will *always* return
immediately even if no data is waiting. *also* openening the pipe for writing
will fix this behaviour.

>How-To-Repeat:

The following snippet of code is stolen from the qmail-1.03 sources and
changed slightly to compile standalone:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/time.h>
#include <unistd.h>

#define FN "temp-trynpbg1.fifo"

void main()
{
  int flagbug;
  struct timeval instant;
  fd_set rfds;
 
  flagbug = 0;
  if (mkfifo(FN,0600) != -1) {
    close(0);
    if (open(FN,O_RDONLY|O_NDELAY) == 0) {
      FD_ZERO(&rfds);
      FD_SET(0,&rfds);
      instant.tv_sec = instant.tv_usec = 0;
      if (select(1,&rfds,(fd_set *) 0,(fd_set *) 0,&instant) > 0)
        flagbug = 1;
    }
    unlink(FN);
  }
  _exit(!flagbug);
}


Compile with cc -o namedpipebug namedpipebug.c
Run with './namedpipebug && echo buggy'

Will echo 'buggy' on FreeBSD, and nothing on, for example, Linux.

>Fix:

The workaround is to also open the socket for writing, as can be seen in this
piece of qmail-1.03 code (from trigger.c):

static int fd = -1;
#ifdef HASNAMEDPIPEBUG1
static int fdw = -1;
#endif
void trigger_set()
{
 if (fd != -1)
   close(fd);
#ifdef HASNAMEDPIPEBUG1
 if (fdw != -1)
   close(fdw);
#endif
 fd = open_read("lock/trigger");
#ifdef HASNAMEDPIPEBUG1
 fdw = open_write("lock/trigger");
#endif
}




>Release-Note:
>Audit-Trail:

From: Sheldon Hearn <sheldonh@uunet.co.za>
To: jhb@FereBSD.org
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: gnu/19871: select bug on named pipes
Date: Fri, 14 Jul 2000 14:40:38 +0200

 Hi John,
 
 Since you've had your fingers in the select code recently, could you
 perhaps check out PR gnu/19871?  (The gnu is inaccurate; the originator
 meant for it to be kern). :-)
 
 Ciao,
 Sheldon.
 

From: Sheldon Hearn <sheldonh@uunet.co.za>
To: jhb@FreeBSD.org
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: gnu/19871: select bug on named pipes
Date: Fri, 14 Jul 2000 14:41:18 +0200

 Hi John,
  
 Since you've had your fingers in the select code recently, could you    
 perhaps check out PR gnu/19871?  (The gnu is inaccurate; the originator 
 meant for it to be kern). :-)
  
 Ciao,
 Sheldon.
 
 

From: John Baldwin <jhb@FreeBSD.org>
To: Sheldon Hearn <sheldonh@uunet.co.za>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: RE: gnu/19871: select bug on named pipes
Date: Fri, 14 Jul 2000 09:20:16 -0700 (PDT)

 On 14-Jul-00 Sheldon Hearn wrote:
 > 
 > Hi John,
 >  
 > Since you've had your fingers in the select code recently, could you    
 > perhaps check out PR gnu/19871?  (The gnu is inaccurate; the originator 
 > meant for it to be kern). :-)
 >  
 > Ciao,
 > Sheldon.
 
 Umm, well, I'm not _that_ familiar with the code, but I could look at
 it I guess.
 
 -- 
 
 John Baldwin <jhb@FreeBSD.org> -- http://www.FreeBSD.org/~jhb/
 PGP Key: http://www.cslab.vt.edu/~jobaldwi/pgpkey.asc
 "Power Users Use the Power to Serve!"  -  http://www.FreeBSD.org/
 

From: Sheldon Hearn <sheldonh@uunet.co.za>
To: John Baldwin <jhb@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: gnu/19871: select bug on named pipes 
Date: Fri, 14 Jul 2000 18:24:05 +0200

 On Fri, 14 Jul 2000 09:20:16 MST, John Baldwin wrote:
 
 > Umm, well, I'm not _that_ familiar with the code, but I could look at
 > it I guess.
 
 I admit, I was chancing my arm.  If you're too busy, let me know and
 I'll reassign the PR.  :-)
 
 Ciao,
 Sheldon.
 

From: "Chris McClellen" <chrismcc@mindspring.com>
To: <bug-followup@freebsd.org>
Cc:  
Subject: Re: kern/19871
Date: Sun, 16 Jul 2000 15:16:23 -0400

 This is a multi-part message in MIME format.
 
 ------=_NextPart_000_001D_01BFEF38.CDC7E5A0
 Content-Type: text/plain;
 	charset="iso-8859-1"
 Content-Transfer-Encoding: 7bit
 
 
 I think I have found the select problem.  I've attached
 a patch to this email against 5.0-current.
 
 Before I explain what is happening, just know that the patch
 is trivial and may break the HELL out of things -- I have 
 _not_ analyzed the impact.  I ran the test in the PR and
 it no longer complains.  However, I have not really tested
 anything else;  even if the patch is bogus, I can still
 explain the problem below:
 
 
 fifo_open (miscfs/fifofs/fifo_vnops.c) looks like it has a 2 stage
 open.
 
 Stage 1 creates read/write sockets, and sets the read socket
 state to SS_CANTRCVMORE.  However, it does _not_ set 
 SS_CANTSENDMORE on the write socket... dont know why.
 
 Stage 2 figures out if you want read/write or both.
 
 When you open for READING, it clears SS_CANTSENDMORE.
 when you open for WRITING, it clears SS_CANTRCVMORE.
 
 It would seem that fifos say "you cant recv if you dont
 have any writers, and you cant send if you dont have any
 readers."
 
 
 Well thats the problem -- select() eventually calls soreadable(), which
 will return true if SS_CANTRCVMORE is set.  If you look in fifo_open
 you'll see its always set until a writer is opened -- thus causing
 select() to always return immediatly until there is a writer, even
 though there is nothing to read.
 
 (When writer is created, the SS_CANTRCVMORE gets cleared, so now
 soreadble will no longer return a false positive).
 
 Since SO_CANTSENDMORE is not set when a writer is open, I figure
 select() will work just fine on a write-only open.  So, you can 
 probably guess the patch -- get rid of SS_CANTRCVMORE on the
 initial open.  I dont know why its there -- and I dont know why
 the same isnt done for the write socket.
 
 Patch is attached, against 5.0-current.  Like I said, it may be bogus
 but at least someone may have enough info now to do a real fix.
 
 Here's the patch inline (its attached because I am almost 
 certain that including it in the email will f it up).  Its 
 included inline so people dont have to get the attachment:
 
 *** miscfs/fifofs/fifo_vnops.c.orig Sun Jul 16 14:47:03 2000
 --- miscfs/fifofs/fifo_vnops.c Sun Jul 16 14:47:23 2000
 ***************
 *** 202,208 ****
     }
     fip->fi_readers = fip->fi_writers = 0;
     wso->so_snd.sb_lowat = PIPE_BUF;
 -   rso->so_state |= SS_CANTRCVMORE;
    }
    if (ap->a_mode & FREAD) {
     fip->fi_readers++;
 --- 202,207 ----
 
 
 ------=_NextPart_000_001D_01BFEF38.CDC7E5A0
 Content-Type: application/octet-stream;
 	name="patch.fifo"
 Content-Transfer-Encoding: quoted-printable
 Content-Disposition: attachment;
 	filename="patch.fifo"
 
 *** miscfs/fifofs/fifo_vnops.c.orig	Sun Jul 16 14:47:03 2000=0A=
 --- miscfs/fifofs/fifo_vnops.c	Sun Jul 16 14:47:23 2000=0A=
 ***************=0A=
 *** 202,208 ****=0A=
   		}=0A=
   		fip->fi_readers =3D fip->fi_writers =3D 0;=0A=
   		wso->so_snd.sb_lowat =3D PIPE_BUF;=0A=
 - 		rso->so_state |=3D SS_CANTRCVMORE;=0A=
   	}=0A=
   	if (ap->a_mode & FREAD) {=0A=
   		fip->fi_readers++;=0A=
 --- 202,207 ----=0A=
 
 ------=_NextPart_000_001D_01BFEF38.CDC7E5A0--
 
 

From: Bruce Evans <bde@zeta.org.au>
To: John Baldwin <jhb@FreeBSD.ORG>
Cc: sheldonh@FreeBSD.ORG
Subject: RE: gnu/19871: select bug on named pipes
Date: Sat, 15 Jul 2000 21:40:08 +1000 (EST)

 On Fri, 14 Jul 2000, John Baldwin wrote:
 
 >  On 14-Jul-00 Sheldon Hearn wrote:
 >  > Since you've had your fingers in the select code recently, could you    
 >  > perhaps check out PR gnu/19871?  (The gnu is inaccurate; the originator 
 >  > meant for it to be kern). :-)
 >  
 >  Umm, well, I'm not _that_ familiar with the code, but I could look at
 >  it I guess.
 
 I fixed read() to on an empty fifo with no writers to conform with
 POSIX.  POSIX requires it to return successfully without blocking.
 In 4.4BSD it has some brokenness related to giving behaviour like the
 author of the PR wants.  select() on read descriptors open on fifos
 just returns when a !O_NONBLOCK read() on one of them would return
 successfully without blocking.  This is always the case for reads on
 a fifo with no writers.  Linux's select() handles fifos specially so
 that select() doesn't just follow read().
 
 SUSv2 specifies select() but doesn't seem to specify anything special
 for pipes.  Therefore, the FreeBSD behaviour is a feature :-).  ISTR
 a previous FR (maybe not in gnats) about this.
 
 FreeBSD has some related bugs in poll().  It only implements POLLHUP for
 ttys and nameless pipes.  Under Linux, POLLHUP is set for polls on a
 fifo with no writers if there was a writer in the (immediately?) previous
 generation, and read-select() checks POLLHUP.  This is necessary to
 give the correct behaviour for read-select() on a fifo whose last writer
 recently went away (select() must not block in this case).
 
 This should be fixed by someone who understands sockets a bit better than
 me.  Named pipes are implemented as sockets, and some of the bugs may be
 for sockets generally.  I think POLLHUP should be set when a socket is
 disconnected...
 
 Bruce
 
 
 

From: Thomas Quinot <thomas@cuivre.fr.eu.org>
To: freebsd-gnats-submit@freebsd.org
Cc: bde@zeta.org.au, jhb@FreeBSD.ORG, sheldonh@FreeBSD.ORG
Subject: Re: kern/19871 select on named pipes always returns 'available for reading'
Date: Tue, 16 Oct 2001 09:44:08 +0200

 I was bitten by this one last night...
 
 While the interpretation of the POSIX definition for select(2) that
 concludes the current FreeBSD behaviour is legal, it may be worth noting
 that Solaris, Irix, AIX, Linux all disagree with us. (On the other hand,
 DEC Unix agrees).
 
 Moreover, the current behaviour means that there is no way of waiting
 for writers to appear on a FIFO with a select. The only possible way
 to do that is to have a dedicated thread blocked in !O_NONBLOCK read.
 
 I would therefore respectfully suggest that we review our implementation
 against the standard (I'll try to grab the copy in the next office later
 today :) ) and see if we can change our current behaviour.
 
 Thomas.
 
 -- 
     Thomas.Quinot@Cuivre.FR.EU.ORG

From: Bruce Evans <bde@zeta.org.au>
To: Thomas Quinot <thomas@cuivre.fr.eu.org>
Cc: <freebsd-gnats-submit@freebsd.org>, <jhb@freebsd.org>,
	<sheldonh@freebsd.org>
Subject: Re: kern/19871 select on named pipes always returns 'available for
 reading'
Date: Tue, 16 Oct 2001 19:30:17 +1000 (EST)

 On Tue, 16 Oct 2001, Thomas Quinot wrote:
 
 > I was bitten by this one last night...
 >
 > While the interpretation of the POSIX definition for select(2) that
 > concludes the current FreeBSD behaviour is legal, it may be worth noting
 > that Solaris, Irix, AIX, Linux all disagree with us. (On the other hand,
 > DEC Unix agrees).
 
 Did any of them document this behaviour?
 
 > Moreover, the current behaviour means that there is no way of waiting
 > for writers to appear on a FIFO with a select. The only possible way
 > to do that is to have a dedicated thread blocked in !O_NONBLOCK read.
 
 True.  I think it should be possible to select() or poll() on an exceptional
 condition.
 
 > I would therefore respectfully suggest that we review our implementation
 > against the standard (I'll try to grab the copy in the next office later
 > today :) ) and see if we can change our current behaviour.
 
 I just checked POSIX-1.200x-draft 7.  It specifies select() but specify
 any special behaviour for select() on pipes or sockets other than it shall
 work.  It just says that select() returns if one of the read descriptors
 is "ready to read".  It doesn't seem to define "ready to read", of course
 %-(.  So my argment for the current behaviour is unchanged.
 
 The draft specifies some special behaviour for poll() on sockets but not
 on pipes.  It says that poll() returns if "data can be read" from one of
 the read descriptors.  It also says that "A file descriptor for a socket
 that is listening for connections shall indicate that it is ready for
 reading, once connections are available".  This corresponds to the blocking
 waiting for writers behaviour that you want.  But it also seems to imply
 returning when a writer appears, whether or not there is anything to read.
 This seems wrong.
 
 Bruce
 

From: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
To: Bruce Evans <bde@zeta.org.au>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/19871 select on named pipes always returns 'available for
 reading'
Date: Tue, 16 Oct 2001 13:01:12 -0400 (EDT)

 <<On Tue, 16 Oct 2001 02:40:02 -0700 (PDT), Bruce Evans <bde@zeta.org.au> said:
 
 >  work.  It just says that select() returns if one of the read descriptors
 >  is "ready to read".  It doesn't seem to define "ready to read", of course
 
 I thought that we had agreed that select would consider ``ready'' as
 meaning ``a read (or write, as appropriate) system call on this file
 descriptor would return without blocking, provided you win any races''.
 
 -GAWollman
 

From: Bruce Evans <bde@zeta.org.au>
To: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
Cc: <freebsd-gnats-submit@FreeBSD.ORG>
Subject: Re: kern/19871 select on named pipes always returns 'available for
 reading'
Date: Wed, 17 Oct 2001 15:38:14 +1000 (EST)

 On Tue, 16 Oct 2001, Garrett Wollman wrote:
 
 > <<On Tue, 16 Oct 2001 02:40:02 -0700 (PDT), Bruce Evans <bde@zeta.org.au> said:
 >
 > >  work.  It just says that select() returns if one of the read descriptors
 > >  is "ready to read".  It doesn't seem to define "ready to read", of course
 >
 > I thought that we had agreed that select would consider ``ready'' as
 > meaning ``a read (or write, as appropriate) system call on this file
 > descriptor would return without blocking, provided you win any races''.
 
 Only for small values of "we" :-).
 
 Bruce
 

From: Thomas Quinot <quinot@inf.enst.fr>
To: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/19871 select on named pipes always returns 'available for reading'
Date: Wed, 17 Oct 2001 14:50:36 +0200

 Le 2001-10-16, Garrett Wollman crivait :
 
 >  I thought that we had agreed that select would consider ``ready'' as
 >  meaning ``a read (or write, as appropriate) system call on this file
 >  descriptor would return without blocking, provided you win any races''.
 
 The question is, is it desirable/feasible to change our definition of
 'ready' to 'a read ... would return /some data (in an amount >0)/
 without blocking ...' ?
 
 Thomas.
 
 -- 
 Thomas Quinot ** Dpartement Informatique & Rseaux ** quinot@inf.enst.fr
               ENST   //   46 rue Barrault   //   75634 PARIS CEDEX 13 
Responsible-Changed-From-To: freebsd-bugs->alfred 
Responsible-Changed-By: alfred 
Responsible-Changed-When: Thu Oct 25 17:38:21 PDT 2001 
Responsible-Changed-Why:  
I'll take care of this. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=19871 

From: "Josh Train" <train@usc.edu>
To: <freebsd-gnats-submit@FreeBSD.org>, <petervd@vuurwerk.nl>
Cc:  
Subject: Re: kern/19871: select on named pipes always returns 'available for reading'
Date: Mon, 10 Jun 2002 14:47:41 -0700

 Hi,
 
 I've run into this problem while trying to use named pipes. I was wondering
 if any progress has been made as far as a resolution to this. It's quite a
 bummer that I cannot add a named pipe to the select and wait for a reader or
 writer to appear.
 
 I would appreciate any info you have on this,
 
 Josh Train
 
State-Changed-From-To: open->closed 
State-Changed-By: alfred 
State-Changed-When: Mon Jun 23 12:34:52 PDT 2003 
State-Changed-Why:  
This is fixed in 5.x, not planning on MFC. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=19871 
>Unformatted:
