From nobody@FreeBSD.org  Tue Jul 13 21:26:45 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 967ED106564A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 13 Jul 2010 21:26:45 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 852658FC1A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 13 Jul 2010 21:26:45 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o6DLQjvV079459
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 13 Jul 2010 21:26:45 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id o6DLQj3q079458;
	Tue, 13 Jul 2010 21:26:45 GMT
	(envelope-from nobody)
Message-Id: <201007132126.o6DLQj3q079458@www.freebsd.org>
Date: Tue, 13 Jul 2010 21:26:45 GMT
From: Manish Vachharajani <manishv@lineratesystems.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: fopen fails with EMFILE if there are more than SHORT_MAX fds open
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         148581
>Category:       kern
>Synopsis:       [libc] fopen(3) fails with EMFILE if there are more than SHORT_MAX fds open
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jul 13 21:30:07 UTC 2010
>Closed-Date:    
>Last-Modified:  Mon Nov 14 18:00:18 UTC 2011
>Originator:     Manish Vachharajani
>Release:        7.3-RELEASE
>Organization:
LineRate Systems
>Environment:
FreeBSD wks1.int.lineratesystems.com 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Fri Jul  2 12:56:26 MDT 2010     root@wks1.int.lineratesystems.com:/usr/obj/usr/src/sys/DEV73PMC  amd64

>Description:
fopen will fail with EMFILE if there are more than SHORT_MAX file descriptors open in the process.  It does not matter that these fds were not created by fopen.  

To make matters worse gcc's libstdc++ uses fopen and friends to implement ofstream so those function mysteriously fail if there are more than 32k fds open in the process.





>How-To-Repeat:
To reproduce the problem compile and run:

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>

#define BIGNUM 50000
int main(int argc, char *argv[]) {
  int i;
  int fds[BIGNUM];
  for(i=0; i < BIGNUM; ++i) {
    fds[i] = open("/dev/null", O_RDONLY);
  }

  FILE *fil = fopen("/dev/null", "r");
  if(fil == NULL) {
    fprintf(stderr, "Could not open /dev/null: %s\n" , strerror(errno));
  }

  for(i=0; i < BIGNUM; ++i) {
    close(fds[i]);
  }

  return 0;
}
>Fix:
The simple fix is to make _file from struct sFILE { ... } FILE; in stdio.h an int instead of a short.  However, this will break binary compatibility with anyone compiled with an old libc.

A very dirty fix that would not break binary compatibility is, for each architecture, use the open space from the padding and alignment requirements of FILE to stash the other bits of _file and make all users of FILE use an accessor macro that pulls out the right bits.

A quick fix to double the threshold at which the problem occurs would be to make _file an unsigned short and use the all 1's value to indicate that this is not a file resource.  Not sure if this will work either, though.

>Release-Note:
>Audit-Trail:

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, manishv@lineratesystems.com
Cc:  
Subject: Re: kern/148581: [libc] fopen(3) fails with EMFILE if there are
 more than SHORT_MAX fds open
Date: Sat, 17 Jul 2010 14:06:00 +0200

 Strictly speaking, your very dirty supposedly safe fix breaks binary
 compatibility because fileno() (in non-threaded programs only) and
 fileno_unlocked() are macros that hard-code the location and size of the
 _file field into binaries. If you have code compiled before the change
 in the same process as code compiled after the change, it might happen
 that data is read/written from/to the wrong descriptor.
 
 What may work is extending FILE (although I'm not entirely sure that
 there is noone that allocates their own FILE) with a 32-bit file
 descriptor field. If the file descriptor exceeds 32767, the 16-bit field
 then contains -1 and fileno() in old binaries will return that. This
 will at least fail safely although fileno() is not defined to return
 error conditions (but it has always returned -1 if the FILE is not
 associated with a file descriptor).
 
 -- 
 Jilles Tjoelker

From: Tobias Oberstein <tobias.oberstein@tavendo.de>
To: "bug-followup@FreeBSD.org" <bug-followup@FreeBSD.org>
Cc:  
Subject: Re: kern/148581: [libc] fopen(3) fails with EMFILE if there are
 more than SHORT_MAX fds open
Date: Mon, 14 Nov 2011 09:47:26 -0800

 Using Manish's test, I could verify that the bug is still present on both i=
 386 and amd64.
 
 FreeBSD XXXXX 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2=
 011     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
 
 FreeBSD XXXXX 8.2-RELEASE-p3 FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:07:27=
  UTC 2011     root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/GENERI=
 C  i386
 
 and even on FreeBSD 9 RC1 !!!!
 
 FreeBSD autobahnhub2 9.0-RC1 FreeBSD 9.0-RC1 #0: Tue Oct 18 18:30:38 UTC 20=
 11     root@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
 
 =3D=3D
 
 I'm doing a kqueue-based network service using Python/Twisted which will ha=
 ppily
 accept >50k TCP connections, but then bails out on Python open(<file>), sin=
 ce
 Python uses fopen(), and
 
 "It does not matter that these fds were not created by fopen."
 
 Python can't be recompiled to use open() (Posix) instead of fopen() (libc).
 
 Only the new Python IO does not use fopen() ... but this leads to other pro=
 blems (for me).
 
 =3D=3D
 
 So this won't be fixed even for FreeBSD 9?
 
 Please ...
 
>Unformatted:
