From nobody@FreeBSD.org  Thu Mar 17 22:38:40 2005
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 81AA616A4CE
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 17 Mar 2005 22:38:40 +0000 (GMT)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 3A62F43D49
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 17 Mar 2005 22:38:40 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j2HMceFf082606
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 17 Mar 2005 22:38:40 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id j2HMce15082605;
	Thu, 17 Mar 2005 22:38:40 GMT
	(envelope-from nobody)
Message-Id: <200503172238.j2HMce15082605@www.freebsd.org>
Date: Thu, 17 Mar 2005 22:38:40 GMT
From: Jakub Kruszona-Zawadzki <acid@gemius.pl>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD freezes on mbufs exhaustion (network interface independent)
X-Send-Pr-Version: www-2.3

>Number:         78968
>Category:       kern
>Synopsis:       FreeBSD freezes on mbufs exhaustion (network interface independent)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-net
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Mar 17 22:40:02 GMT 2005
>Closed-Date:    
>Last-Modified:  Tue Jun 15 17:45:51 UTC 2010
>Originator:     Jakub Kruszona-Zawadzki
>Release:        FreeBSD 5.3
>Organization:
Gemius SA
>Environment:
FreeBSD router.gucio.pl 5.3-STABLE FreeBSD 5.3-STABLE #4: Sun Dec  5 04:53:26 CET 2004     root@router.gucio.pl:/usr/obj/usr/src/sys/GENERIC  i386      
>Description:
We are using all of our output bandwidth. Tcp stack allocates mbufs for output queues. When mbufs space is exhausted system freezes. Even 'numlock' on keyboard don't work. It looks like kernel is going into an infinite loop or something - system is dead and the only thing you can do is reboot.
>How-To-Repeat:
Set max mbufs to a small number (only for testing).
Generate a file about 300kB (i.e. dd if=/dev/urandom of=/tmp/file bs=1024 count=300).
Add to inetd.conf line like this:
telnet  stream  tcp     nowait  root    /bin/cat        cat     /tmp/file
Restart inetd with these options (maybe not all are necessary):
inetd -c 0 -C 0 -R 0 -s 0

On remote machine run many processes that will open many connections to your serwer on telent port, but WILL NOT download the file. You can use these programs:
In python:
#!/usr/bin/env python
from socket import socket
from time import sleep

socks = []
for i in xrange(240):
    try:
        s = socket()
        s.connect(('192.168.0.1',23))
        socks.append(s)
    except Exception:
        pass

print "opened"
sleep(100)

for i in socks:
    i.close()

In php:
<?
 for ($i=0;$i<1000;$i++) {
  if(!($file[$i]=fsockopen('192.168.0.1',23)))
   break;
  fgets($file[$i],1);
 }
?>

FreeBSD 5.3 with default number of mbufs (about 17000) freezes after a few minutes, when both these programs work in several (6 pythons an 15 phps) copies.
While these programs are active, "netstat -n" should be showing many "ESTABLISHED" connections with full output queue. In "netstat -m" output number of "mbufs in use" and "mbuf clusters in use" should increase. When current number of "mbuf clusters in use" reaches max value, system freezes in a few seconds.

During normal work "netstat -m" looks like this:
334 mbufs in use
198/17088 mbuf clusters in use (current/max)
0/3/4528 sfbufs in use (current/peak/max)
479 KBytes allocated to network
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
104 calls to protocol drain routines

>Fix:
This problem doesn't exist in FreeBSD 4.x
>Release-Note:
>Audit-Trail:

From: Ernie Smallis <esmallis@stbernard.com>
To: bug-followup@freebsd.org, acid@gemius.pl
Cc:  
Subject: Re: kern/78968: FreeBSD freezes on mbufs exhaustion (network interface independent)
Date: Tue, 17 May 2005 11:41:47 -0700

 I am seeing this too:
 
 The fxp driver is receiving a return of ENOBUFS from m_getcl() (no surprise 
 there).  However, since there are no clusters available and the fxp driver 
 has frames to DMA from its FIFO; it keeps calling for clusters which never 
 happens successfully.  It appears the system is hung but; in fact, it is in a 
 tight loop.
 
 I believe BSD 4.x had a patch that fixed a panic for the exhaustion situation 
 since the code would be de-referencing a NULL pointer.  So, now we have a 
 test for NULL and the return of ENOBUFS.  That's one way of not seeing the 
 current problem ;-)
 
 Another way is to fix this.  The test for NULL is certainly appropriate 
 however, what happens next isn't.   I am looking to contact the relevant  
 committer(s) for a discussion as to how to resolve this problem.
 
 Regards,
 
 Ernie ;-)
Responsible-Changed-From-To: freebsd-bugs->gnn 
Responsible-Changed-By: gnn 
Responsible-Changed-When: Sun Nov 12 09:52:00 UTC 2006 
Responsible-Changed-Why:  
This shoiuld be verified or closed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=78968 
Responsible-Changed-From-To: gnn->freebsd-net 
Responsible-Changed-By: gnn 
Responsible-Changed-When: Tue Jun 15 17:35:12 UTC 2010 
Responsible-Changed-Why:  
5.3 bug, probably no longer relevant. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=78968 
>Unformatted:
