From lennox@cs.columbia.edu  Wed Oct 29 08:31:49 2003
Return-Path: <lennox@cs.columbia.edu>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 89A4716A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 29 Oct 2003 08:31:49 -0800 (PST)
Received: from cs.columbia.edu (cs.columbia.edu [128.59.16.20])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 07D6843FAF
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 29 Oct 2003 08:31:48 -0800 (PST)
	(envelope-from lennox@cs.columbia.edu)
Received: from cnr.cs.columbia.edu (cnr.cs.columbia.edu [128.59.19.133])
	by cs.columbia.edu (8.12.10/8.12.10) with ESMTP id h9TGVkLa027604
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 29 Oct 2003 11:31:46 -0500 (EST)
Received: from cnr.cs.columbia.edu (localhost [127.0.0.1])
	by cnr.cs.columbia.edu (8.12.9p1/8.12.9) with ESMTP id h9TGVj2Y005482
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 29 Oct 2003 11:31:45 -0500 (EST)
	(envelope-from lennox@cnr.cs.columbia.edu)
Received: (from lennox@localhost)
	by cnr.cs.columbia.edu (8.12.9p1/8.12.9/Submit) id h9TGVjLu005481;
	Wed, 29 Oct 2003 11:31:45 -0500 (EST)
Message-Id: <200310291631.h9TGVjLu005481@cnr.cs.columbia.edu>
Date: Wed, 29 Oct 2003 11:31:45 -0500 (EST)
From: Jonathan Lennox <lennox@cs.columbia.edu>
Reply-To: Jonathan Lennox <lennox@cs.columbia.edu>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: gethostbyname leaks kqueue file descriptors with pthreads and static linking
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         58687
>Category:       kern
>Synopsis:       [libc] [patch] gethostbyname(3) leaks kqueue file descriptors with pthreads and static linking
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    deischen
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Oct 29 08:40:20 PST 2003
>Closed-Date:    Tue Nov 15 16:52:56 UTC 2011
>Last-Modified:  Tue Nov 15 16:52:56 UTC 2011
>Originator:     Jonathan Lennox
>Release:        FreeBSD 5.1-RELEASE-p10 i386
>Organization:
Columbia University
>Environment:
System: FreeBSD cnr.cs.columbia.edu 5.1-RELEASE-p10 FreeBSD 5.1-RELEASE-p10 #8: Sun Oct 5 23:47:09 EDT 2003 lennox@cnr.cs.columbia.edu:/usr/obj/usr/src/sys/CNR i386

The system has been patched by adding
src/lib/libc_r/uthread/uthread_kqueue.c, aligning it with -CURRENT.  (This
is mp's patch of 2003-07-25.)

>Description:

When linking with a static libc_r and libc, gethostbyname() leaks kqueue
file descriptors.  (I actually first observed this with libc_r_p.a and
libc_p.a, i.e. compiling and linking with -pg, but the same thing happens
when linking with -static.)

This is because &_kqueue isn't included in the 'references' array in
uthread_init.c.  There's no outstanding unresolved link to _kqueue when
libc_r is linked in, so the linker only picks up the version from libc.

See also closed PRs kern/55007 and bin/55879, which are this same problem
with dynamic libraries, now fixed in -CURRENT and -STABLE.

>How-To-Repeat:

Compile the following program with static libc and libc_r, either with
-static or -pg:

----------
#include <stdio.h>
#include <pthread.h>
#include <netdb.h>

void* the_thread(void* dummy)
{
  int i;
  for (i = 0; i < 50; i++) {
    gethostbyname("www.freebsd.org");
  } 

  return NULL;
}

int main(void)
{
  pthread_t dummy;
  pthread_create(&dummy, NULL, &the_thread, NULL);

  getchar();

  return 0;
}
----------

Before the program exits, use lsof to observe its open file descriptors.
Notice that it has 50 KQUEUE descriptors open.

In -CURRENT, this problem does not occur if you use dynamic libc and libc_r,
though it does in 5.1-RELEASE.  


>Fix:

Apply the following patch and rebuild:

--- lib/libc_r/uthread/uthread_init.c.orig	Wed Oct 29 11:00:53 2003
+++ lib/libc_r/uthread/uthread_init.c	Wed Oct 29 11:01:21 2003
@@ -99,6 +99,7 @@
 	&_getsockopt,
 	&_ioctl,
 	&_kevent,
+	&_kqueue,
 	&_listen,
 	&_nanosleep,
 	&_open,


As a workaround:

Add the command-line option -u kqueue (-Wl,-u,kqueue from gcc) when
linking.
>Release-Note:
>Audit-Trail:

From: Jonathan Lennox <lennox@cs.columbia.edu>
To: freebsd-gnats-submit@FreeBSD.org, lennox@cs.columbia.edu
Cc:  
Subject: Re: bin/58687: gethostbyname leaks kqueue file descriptors with pthreads and static linking
Date: Wed, 29 Oct 2003 14:58:51 -0500

 Apologies, the previous patch was incomplete.  This is the correct patch,
 which has been built, installed, and confirmed to solve the problem.
 
 --- lib/libc_r/uthread/uthread_init.c.orig	Wed Oct 29 11:00:53 2003
 +++ lib/libc_r/uthread/uthread_init.c	Wed Oct 29 11:01:21 2003
 @@ -99,6 +99,7 @@
  	&_getsockopt,
  	&_ioctl,
  	&_kevent,
 +	&_kqueue,
  	&_listen,
  	&_nanosleep,
  	&_open,
 --- lib/libc/include/namespace.h.orig	Wed Oct 29 14:13:09 2003
 +++ lib/libc/include/namespace.h	Wed Oct 29 14:13:31 2003
 @@ -77,6 +77,7 @@
  #define		getsockopt			_getsockopt
  #define		ioctl				_ioctl
  /* #define		kevent				_kevent */
 +#define		kqueue				_kqueue
  #define		listen				_listen
  #define		nanosleep			_nanosleep
  #define		open				_open
 --- lib/libc/include/un-namespace.h.orig	Wed Oct 29 14:13:13 2003
 +++ lib/libc/include/un-namespace.h	Wed Oct 29 14:13:55 2003
 @@ -58,6 +58,7 @@
  #undef		getsockopt
  #undef		ioctl
  #undef		kevent
 +#undef		kqueue
  #undef		listen
  #undef		nanosleep
  #undef		open

From: Kris Kennaway <kris@obsecurity.org>
To: Jonathan Lennox <lennox@cs.columbia.edu>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: bin/58687: gethostbyname leaks kqueue file descriptors with pthreads and static linking
Date: Thu, 30 Oct 2003 01:03:51 -0800

 On Wed, Oct 29, 2003 at 11:31:45AM -0500, Jonathan Lennox wrote:
 
 > In -CURRENT, this problem does not occur if you use dynamic libc and libc_r,
 > though it does in 5.1-RELEASE.  
 
 I'm confused..are you saying the problem does not exist in -CURRENT,
 or it still does in restricted circumstances?
 
 Kris

From: Jonathan Lennox <lennox@cs.columbia.edu>
To: Kris Kennaway <kris@obsecurity.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: bin/58687: gethostbyname leaks kqueue file descriptors with pthreads and static linking
Date: Thu, 30 Oct 2003 09:20:06 -0500

 On Thursday, October 30 2003, Kris Kennaway wrote:
 
 > On Wed, Oct 29, 2003 at 11:31:45AM -0500, Jonathan Lennox wrote:
 > 
 > > In -CURRENT, this problem does not occur if you use dynamic libc and libc_r,
 > > though it does in 5.1-RELEASE.  
 > 
 > I'm confused..are you saying the problem does not exist in -CURRENT,
 > or it still does in restricted circumstances?
 
 In -CURRENT it exists in restricted circumstances -- i.e. it only occurs if
 you're using a static libc and libc_r, because of the -pg or -static flags
 to gcc.
 
 In 5.1-RELEASE it occurs any time you're using libc_r.  This was fixed by
 mp@FreeBSD.org with a patch on 2003-07-25, but the fix doesn't work for the
 static library case.  My patch extends mp's fix so it covers the static
 library case as well.

From: Jonathan Lennox <lennox@cs.columbia.edu>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: bin/58687: gethostbyname leaks kqueue file descriptors with pthreads and static linking
Date: Tue, 13 Jan 2004 17:09:40 -0500

 Update: this problem is still present in FreeBSD 5.2-RELEASE.
 
 I.e., compiling the code above with
       gcc -Wall -g -O2 -pg kqueue.c -o kqueue -pthread
 or
       gcc -Wall -g -O2 -static kqueue.c -o kqueue -pthread
 
 causes the kqueue executable to hold 50 KQUEUE file descriptors.
 
 The patch still applies cleanly.
Responsible-Changed-From-To: freebsd-bugs->deischen 
Responsible-Changed-By: green 
Responsible-Changed-When: Tue Apr 27 19:07:13 PDT 2004 
Responsible-Changed-Why:  
Dan has been handling these thread-library things. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=58687 

From: Bryan Drewery <bryan@xzibition.com>
To: bug-followup@FreeBSD.org,  lennox@cs.columbia.edu
Cc:  
Subject: Re: kern/58687: [libc] [patch] gethostbyname(3) leaks kqueue file
 descriptors with pthreads and static linking
Date: Mon, 08 Dec 2008 17:08:25 -0600

 I still have this problem to date on FreeBSD 6.x and 7.0.
 
 The patch given in the PR never made it into uthread/libc_r it seems.
 
 My problem specifically is that non-threaded dynamic applications which
 use setjmp to jump out of gethostbyname2() leave behind a KQUEUE
 descriptor which is then counted against the user's FD limits set by uname.
 
 This happens when the DNS server cannot be contacted before the alarm is
 signaled.
 
 Hoping this gets fixed,
 Bryan Drewery
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/58687: commit references a PR
Date: Sun,  8 Feb 2009 06:45:13 +0000 (UTC)

 Author: deischen
 Date: Sun Feb  8 06:44:58 2009
 New Revision: 188300
 URL: http://svn.freebsd.org/changeset/base/188300
 
 Log:
   Fix leak of kqueue() file descriptors when linked with static
   libc_r.
   
   PR:		58687
   Submitted by:	Jonathon Lennox <lennox at cs dot columbia dot edu>
 
 Modified:
   head/lib/libc_r/uthread/uthread_init.c
 
 Modified: head/lib/libc_r/uthread/uthread_init.c
 ==============================================================================
 --- head/lib/libc_r/uthread/uthread_init.c	Sun Feb  8 03:03:55 2009	(r188299)
 +++ head/lib/libc_r/uthread/uthread_init.c	Sun Feb  8 06:44:58 2009	(r188300)
 @@ -95,6 +95,7 @@ static void *references[] = {
  	&_getsockopt,
  	&_ioctl,
  	&_kevent,
 +	&_kqueue,
  	&_listen,
  	&_nanosleep,
  	&_open,
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched  
State-Changed-By: brucec 
State-Changed-When: Mon Mar 23 21:43:09 UTC 2009 
State-Changed-Why:  
Fixed in -current. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=58687 

From: Niclas Zeising <niclas.zeising@gmail.com>
To: bug-followup@FreeBSD.org, lennox@cs.columbia.edu
Cc: deischen@freebsd.org
Subject: Re: kern/58687: [libc] [patch] gethostbyname(3) leaks kqueue file
 descriptors with pthreads and static linking
Date: Tue, 17 May 2011 16:12:19 +0200

 Hi!
 From what I can tell from the patch in the PR and what has been
 committed, the wrong patch was committed. The patch included changes to
 lib/libc/include/namespace.h and un-namespace.h as well. This is no
 problem for current, where libc_r is removed though.
 Can this be MFCd, or does it change ABI/API somehow? If it doesn't, it
 might be merged to 7-STABLE (It's already in 8) and the PR closed.
 Regards!
 -- 
 Niclas

From: Daniel Eischen <eischen@vigrid.com>
To: Niclas Zeising <niclas.zeising@gmail.com>
Cc: "bug-followup@FreeBSD.org" <bug-followup@FreeBSD.org>,
        "lennox@cs.columbia.edu" <lennox@cs.columbia.edu>,
        "deischen@FreeBSD.org" <deischen@FreeBSD.org>
Subject: Re: kern/58687: [libc] [patch] gethostbyname(3) leaks kqueue file descriptors with pthreads and static linking
Date: Tue, 17 May 2011 12:31:49 -0500

 On May 17, 2011, at 9:12 AM, Niclas Zeising <niclas.zeising@gmail.com> wrote=
 :
 
 > Hi!
 > =46rom what I can tell from the patch in the PR and what has been
 > committed, the wrong patch was committed. The patch included changes to
 > lib/libc/include/namespace.h and un-namespace.h as well. This is no
 > problem for current, where libc_r is removed though.
 > Can this be MFCd, or does it change ABI/API somehow? If it doesn't, it
 > might be merged to 7-STABLE (It's already in 8) and the PR closed.
 > Regards!
 
 The change to [un-]namespace.h is incorrect because there is a struct kqueue=
 .  Same with kevent, see how kevent is commented out in namespace.h.
 
 --
 DE=

From: Niclas Zeising <niclas.zeising@gmail.com>
To: Daniel Eischen <eischen@vigrid.com>
Cc: "bug-followup@FreeBSD.org" <bug-followup@FreeBSD.org>, 
 "lennox@cs.columbia.edu" <lennox@cs.columbia.edu>,
 "deischen@FreeBSD.org" <deischen@FreeBSD.org>
Subject: Re: kern/58687: [libc] [patch] gethostbyname(3) leaks kqueue file
 descriptors with pthreads and static linking
Date: Tue, 17 May 2011 20:06:16 +0200

 On 2011-05-17 19:31, Daniel Eischen wrote:
 > On May 17, 2011, at 9:12 AM, Niclas Zeising <niclas.zeising@gmail.com> wrote:
 > 
 >> Hi!
 >> From what I can tell from the patch in the PR and what has been
 >> committed, the wrong patch was committed. The patch included changes to
 >> lib/libc/include/namespace.h and un-namespace.h as well. This is no
 >> problem for current, where libc_r is removed though.
 >> Can this be MFCd, or does it change ABI/API somehow? If it doesn't, it
 >> might be merged to 7-STABLE (It's already in 8) and the PR closed.
 >> Regards!
 > 
 > The change to [un-]namespace.h is incorrect because there is a struct kqueue.  Same with kevent, see how kevent is commented out in namespace.h.
 > 
 > --
 > DE
 
 Ok, thanks for the clarification. I still think that the patch can be
 merged to 7-stable and the PR closed though.
 Regards!
 -- 
 Niclas
State-Changed-From-To: patched->closed 
State-Changed-By: eadler 
State-Changed-When: Tue Nov 15 16:52:55 UTC 2011 
State-Changed-Why:  
this was commited a long time ago 

http://www.freebsd.org/cgi/query-pr.cgi?pr=58687 
>Unformatted:
