From nobody@FreeBSD.org  Mon Sep 21 01:12:25 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id E46CE106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 21 Sep 2009 01:12:25 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id D373B8FC0C
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 21 Sep 2009 01:12:25 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n8L1CPnh051036
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 21 Sep 2009 01:12:25 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n8L1CPbT051035;
	Mon, 21 Sep 2009 01:12:25 GMT
	(envelope-from nobody)
Message-Id: <200909210112.n8L1CPbT051035@www.freebsd.org>
Date: Mon, 21 Sep 2009 01:12:25 GMT
From: Jacob Myers <jacob@whotookspaz.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Lighttpd/php-cgi with freebsd-sendfile enabled causing kernel to not reenter userland
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         138999
>Category:       kern
>Synopsis:       [libc] lighttpd/php-cgi with freebsd sendfile(2) enabled causing kernel to not reenter userland
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-net
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Sep 21 01:20:01 UTC 2009
>Closed-Date:    Sat Jan 23 19:03:18 UTC 2010
>Last-Modified:  Sat Jan 23 19:03:18 UTC 2010
>Originator:     Jacob Myers
>Release:        7.2-RELEASE-p3
>Organization:
Wilcox Technologies
>Environment:
FreeBSD IND-Serv003.Wilcox-Tech.com 7.2-RELEASE-p3 FreeBSD 7.2-RELEASE-p3 #1: Wed Aug 26 23:16:03 EDT 2009     root@IND-Serv003.Wilcox-Tech.com:/usr/obj/usr/src/sys/IND-SERV003  i386
>Description:
There is some sort of odd issue involving FreeBSD and lighttpd with
respect to lighttpd's use of sendfile(2) causing the kernel to not
return to userland (or so we suspect). The kernel responds to ping (and
even rate limits ping floods), but all SSH and other connections stop,
and all login attempts at the console hang.

I have investigated this problem with a friend of mine, and have been
unable to draw any conclusions. Testing this problem with debugging
symbols compiled in has caused the bug to manifest itself in a new way,
in that sendfile(2) continuously returns EAGAIN without end.
>How-To-Repeat:
0) Make sure the kernel has no debugging enabled (e.g., witness, etc.)
1) Run lighttpd with php-cgi
2) Set server.network-backend = freebsd-sendfile in lighttpd.conf
3) Attempt to upload a file > 2 MB in length or so using a simple PHP script
4) The machine should then be locked from new logins, but will respond to ping
>Fix:
None known.

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Mon Sep 21 11:22:34 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s).  To submitter: can you share your debugging 
output?  And also perhaps the example PHP sript so that people who 
don't necessarily write PHP can recreate the bug?  Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=138999 

From: Jacob Myers <jacob@whotookspaz.org>
To: bug-followup@FreeBSD.org, jacob@whotookspaz.org
Cc:  
Subject: Re: kern/138999: [libc] lighttpd/php-cgi with freebsd sendfile(2)
 enabled causing kernel to not reenter userland
Date: Thu, 24 Sep 2009 05:48:54 -0400

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Well, I'd love to give you some debugging output, but unfortunately we
 never got the system to panic or hang with debugging symbols. I'm not
 entirely sure what sort of info would be useful for it returning EINVAL
 repeatedly...
 
 As for the PHP scripts you requested, they're attached (use upload.php
 first. Since I figured it'd be useful, I also attached a sample
 lighttpd.conf.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.13 (FreeBSD)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
 iQIcBAEBAgAGBQJKuz+8AAoJEA933foYakKkd34P/2/Uwgvhik8OJdZr0py4xTAp
 J3d50d+QXDtiUZSrjYCBePsmp1tDy7dnIQ1YjodbjFks5xAS0+VlfUIoq8/Ks0jn
 g0LOvpm0NaPBszO+9rTQTum84HyoGrHGlOrMbz+qNTSOQiv2wLvMQDUh3AuGun4N
 SXZH3S1p7thKYIf2cPw58A0t8/KB6Jxljeg35lw+lpaF88ZOSOte5Veq/If/Enx+
 7Svtp90Zt6yKBeMwSOSDHIgQCqo9rKzBzBg2K5k8xA7i8zeDr2Ee3dD7fIuUBdwS
 2iUyNqFDxlihmTs+II/p/cf6b4o8AqDuVuI99j/vUcS4FWnyAzusSgvTo6iNPk1w
 1FkCEak07l79HrlmYpubMSfV2gusSjgL9Y7xmLt9FPULQeoJPix71LF+Ch+unjAz
 Q0+NqywJif+lre7a89MMdFPyI4OLgV7BfwNose6SpOM1I7W2uV6IVWABYCnLpT0l
 5K6taF1GnPnX6NlXMh31AoIU8/WTshay6RJnnbqV4i4bE+nBTR5tHsr6Do41Wuid
 sC5xtiZ6nOxRxR0e+Wv8nn2i6+A1lbgf2SBM1XmuxFpObzD+saPJKNP0pBP+5iEJ
 O1ccrRzRgWy+ndkIUDAhaOeSnswDK3cRmNnhYchhLq95qLYJoSKRriNmaqm+378U
 CxOa0nUyY9GAuMs/bJAl
 =GCEq
 -----END PGP SIGNATURE-----

From: Jacob Myers <jacob@whotookspaz.org>
To: bug-followup@FreeBSD.org, jacob@whotookspaz.org
Cc:  
Subject: Re: kern/138999: [libc] lighttpd/php-cgi with freebsd sendfile(2)
 enabled causing kernel to not reenter userland
Date: Thu, 24 Sep 2009 05:51:38 -0400

 This is a multi-part message in MIME format.
 --------------040709090100010605060305
 Content-Type: text/plain; charset=ISO-8859-1
 Content-Transfer-Encoding: 7bit
 
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Er, whoops.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.13 (FreeBSD)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
 
 iQIcBAEBAgAGBQJKu0EoAAoJEA933foYakKkoi0P/0eZ3z9frqhqacfcgwJEtXro
 6e+cVyqdUp4LsQuh4198jrhuA6IXxAHVUB6JP5wkThA7ml0h5Jwd4w/LV2fPlFvC
 F8rYxyLZvrHenUWJoJELyZLt5B4P6QgwLD6+l1nyW6vE4IZZmnfqX9nqbE101xq8
 tBcx1THXWAGgd+nrswDfKv5Y7QPzMjdCGA5vfnCQ9TQREoqgvCz0kuWKfHE3SpFc
 Pg2d/fyUwHNXOqJbU2N+HLqht+MT5wL/SAitDd5h4sUUtzi9cQ3h1niHwed+ctTP
 evsek0wdmfaVcdURNtLGYDos7OAszdVF2yKZO+I2IYiOhDZnGtVrLxM1BPYIcT7R
 xnUzIw5/A/u3raNpfkmHKbZkqOLZhu4IqFWmdLUjzKo/LvNoEMsIwIU7Zdcxro7s
 gYo5hGw9y2TNBQVOuZLC1/UN6vvAM7/MwL/AdVmP1wbyShg3t6n3pez/o95fy/Iq
 4uTqYqQFgbFAZtURXnVm/0x5TjZJKvlSUcE7ckzRhzNOifaO4mEyEIHIPd0EIkdd
 qqzQ2OGnAUFXkRqCj15CIDx/qLo5O0jhmy9cCBX/BO4AVwvBXEdPMLl2x9IBTizr
 N5QIvEPUY3X9LNZgE5jcbt1hnSq1fmurkfX/ECJ1i4NAv4NhWLGEmeEwrWRtUIin
 DzvqkA+brIyLdZKGTmq0
 =X+6G
 -----END PGP SIGNATURE-----
 
 --------------040709090100010605060305
 Content-Type: text/plain;
  name="sendfile.sh"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline;
  filename="sendfile.sh"
 
 # This is a shell archive.  Save it in a file, remove anything before
 # this line, and then unpack it by entering "sh file".  Note, it may
 # create directories; files and directories will be owned by you and
 # have default permissions.
 #
 # This archive contains:
 #
 #	sendfile/lighttpd.conf
 #	sendfile/upload.php
 #	sendfile/uploader.php
 #
 echo x - sendfile/lighttpd.conf
 sed 's/^X//' >sendfile/lighttpd.conf << '13d6c5353933d78bbe99686dc1205d95'
 Xserver.modules              = ( "mod_access",               
 X                                "mod_fastcgi")
 X
 Xserver.document-root        = "/usr/local/www/default/"
 Xserver.errorlog             = "/var/log/lighttpd/default/error.log"
 X
 Xindex-file.names            = ( "index.php", "index.html",
 X                                "index.htm", "default.htm" )
 X
 Xserver.event-handler = "freebsd-kqueue"
 Xserver.network-backend = "freebsd-sendfile"
 X
 Xmimetype.assign             = (
 X  ".pdf"          =>      "application/pdf",
 X  ".sig"          =>      "application/pgp-signature",
 X  ".spl"          =>      "application/futuresplash", 
 X  ".class"        =>      "application/octet-stream", 
 X  ".ps"           =>      "application/postscript",   
 X  ".torrent"      =>      "application/x-bittorrent", 
 X  ".dvi"          =>      "application/x-dvi",        
 X  ".gz"           =>      "application/x-gzip",       
 X  ".pac"          =>      "application/x-ns-proxy-autoconfig",
 X  ".swf"          =>      "application/x-shockwave-flash",    
 X  ".tar.gz"       =>      "application/x-tgz",                
 X  ".tgz"          =>      "application/x-tgz",                
 X  ".tar"          =>      "application/x-tar",                
 X  ".zip"          =>      "application/zip",                  
 X  ".mp3"          =>      "audio/mpeg",                       
 X  ".m3u"          =>      "audio/x-mpegurl",                  
 X  ".wma"          =>      "audio/x-ms-wma",                   
 X  ".wax"          =>      "audio/x-ms-wax",                   
 X  ".ogg"          =>      "application/ogg",                  
 X  ".wav"          =>      "audio/x-wav",                      
 X  ".gif"          =>      "image/gif",                        
 X  ".jar"          =>      "application/x-java-archive",       
 X  ".jpg"          =>      "image/jpeg",                       
 X  ".jpeg"         =>      "image/jpeg",                       
 X  ".png"          =>      "image/png",                        
 X  ".xbm"          =>      "image/x-xbitmap",                  
 X  ".xpm"          =>      "image/x-xpixmap",                  
 X  ".xwd"          =>      "image/x-xwindowdump",              
 X  ".css"          =>      "text/css",                         
 X  ".html"         =>      "text/html",                        
 X  ".htm"          =>      "text/html",                        
 X  ".js"           =>      "text/javascript",                  
 X  ".asc"          =>      "text/plain",                       
 X  ".c"            =>      "text/plain",                       
 X  ".cpp"          =>      "text/plain",                       
 X  ".log"          =>      "text/plain",                       
 X  ".conf"         =>      "text/plain",                       
 X  ".text"         =>      "text/plain",                       
 X  ".txt"          =>      "text/plain",                       
 X  ".diff"         =>      "text/plain",                       
 X  ".patch"        =>      "text/plain",                       
 X  ".dtd"          =>      "text/xml",                         
 X  ".xml"          =>      "text/xml",                         
 X  ".mpeg"         =>      "video/mpeg",                       
 X  ".mpg"          =>      "video/mpeg",                       
 X  ".mov"          =>      "video/quicktime",                  
 X  ".qt"           =>      "video/quicktime",                  
 X  ".avi"          =>      "video/x-msvideo",                  
 X  ".asf"          =>      "video/x-ms-asf",                   
 X  ".asx"          =>      "video/x-ms-asf",                   
 X  ".wmv"          =>      "video/x-ms-wmv",                   
 X  ".bz2"          =>      "application/x-bzip",               
 X  ".tbz"          =>      "application/x-bzip-compressed-tar",
 X  ".tar.bz2"      =>      "application/x-bzip-compressed-tar",
 X  # default mime type                                         
 X  ""              =>      "application/octet-stream",         
 X )                                                            
 X
 Xmimetype.use-xattr        = "enable"                                       
 X
 Xaccesslog.filename          = "/var/log/lighttpd/default/access.log"
 X
 Xstatic-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )        
 X
 Xserver.pid-file            = "/var/run/lighttpd.pid"
 X
 Xserver.username            = "www"          
 Xserver.groupname           = "www"          
 X
 Xfastcgi.server             = ( ".php" =>                          
 X                               ( "localhost" =>                   
 X                                 (                                
 X                                   "socket" => "/var/run/lighttpd/php-fastcgi.socket",
 X                                   "bin-path" => "/usr/local/bin/php-cgi",            
 X                                   "max-procs" => 2                                   
 X                                 )                                                    
 X                               )                                                      
 X                            )                                                         
 X
 X
 Xserver.stat-cache-engine = "fam"
 X
 Xserver.use-ipv6 = "enable"     
 X$SERVER["socket"] == "0.0.0.0:80" { }
 13d6c5353933d78bbe99686dc1205d95
 echo x - sendfile/upload.php
 sed 's/^X//' >sendfile/upload.php << '5519f8f464a953c27a762163cfade428'
 X<html>
 X	<title>Upload thingy</title>	
 X	</head>
 X
 X	<body>
 X		<div id="container">
 X			<div id="mainbody">
 X			<br />
 X				<form enctype="multipart/form-data" action="uploader.php" method="POST">
 X					<h2>Choose a flash thingy to upload:</h2>
 X					<br />
 X					<input name="uploadedfile" type="file"/><br />
 X					<input type="submit" value="Upload"/>
 X				</form>
 X			</div>
 X		</div>
 X	</body>
 X</html>
 X
 5519f8f464a953c27a762163cfade428
 echo x - sendfile/uploader.php
 sed 's/^X//' >sendfile/uploader.php << 'fe7d407dda27de3d9cfe521e27e7a469'
 X<html>						     
 X	<head>
 X		<title>Upload!</title>
 X	</head>
 X
 X	<body>
 X		<div id="container">
 X			<div id="mainbody">
 X			<br />	     
 X<?php				      
 X	function verify_file()	     
 X	{				  
 X		// Do we even have a file? 
 X		if ($_FILES['uploadedfile'] == NULL)
 X		{				   
 X			return "No file!";    
 X		}				   
 X
 X		if ($_FILES['uploadedfile']['size'] < 1048576)
 X		{
 X			return "It's not big enough to trigger the bug";
 X		}
 X
 X		// Where the file is going to be placed
 X		$target_path = "junk/";
 X
 X		// Add the original filename to our target path.
 X		$target_path = $target_path . basename( $_FILES['uploadedfile']['name']);
 X
 X		if (file_exists($target_path))
 X		{
 X			return "This file already exists.";
 X		}
 X
 X		if (move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target_path))
 X		{
 X			return "The file ".  basename( $_FILES['uploadedfile']['name']).
 X				" has been uploaded";
 X		}
 X		else
 X		{
 X			return "There was an error uploading the file.";
 X		}
 X	}
 X
 X	$rvalue = verify_file();
 X	echo "			  <h2>$rvalue</h2>\n";
 X?>
 X			</div>
 X		</div>
 X	</body>
 X</html>
 X
 fe7d407dda27de3d9cfe521e27e7a469
 exit
 
 
 --------------040709090100010605060305--

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/138999: commit references a PR
Date: Tue,  3 Nov 2009 12:52:49 +0000 (UTC)

 Author: kib
 Date: Tue Nov  3 12:52:35 2009
 New Revision: 198853
 URL: http://svn.freebsd.org/changeset/base/198853
 
 Log:
   If socket buffer space appears to be lower then sum of count of already
   prepared bytes and next portion of transfer, inner loop of kern_sendfile()
   aborts, not preparing next mbuf for socket buffer, and not modifying
   any outer loop invariants. The thread loops in the outer loop forever.
   
   Instead of breaking from inner loop, prepare only bytes that fit into
   the socket buffer space.
   
   In collaboration with:	pho
   Reviewed by:	bz
   PR:	kern/138999
   MFC after:	2 weeks
 
 Modified:
   head/sys/kern/uipc_syscalls.c
 
 Modified: head/sys/kern/uipc_syscalls.c
 ==============================================================================
 --- head/sys/kern/uipc_syscalls.c	Tue Nov  3 12:03:13 2009	(r198852)
 +++ head/sys/kern/uipc_syscalls.c	Tue Nov  3 12:52:35 2009	(r198853)
 @@ -2037,20 +2037,12 @@ retry_space:
  				rem = obj->un_pager.vnp.vnp_size -
  				    uap->offset - fsbytes - loopbytes;
  			xfsize = omin(rem, xfsize);
 +			xfsize = omin(space - loopbytes, xfsize);
  			if (xfsize <= 0) {
  				VM_OBJECT_UNLOCK(obj);
  				done = 1;		/* all data sent */
  				break;
  			}
 -			/*
 -			 * Don't overflow the send buffer.
 -			 * Stop here and send out what we've
 -			 * already got.
 -			 */
 -			if (space < loopbytes + xfsize) {
 -				VM_OBJECT_UNLOCK(obj);
 -				break;
 -			}
  
  			/*
  			 * Attempt to look up the page.  Allocate
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: kib 
State-Changed-When: Sat Jan 23 19:02:18 UTC 2010 
State-Changed-Why:  
Patch is already in HEAD and stable/8, stable/7. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=138999 
>Unformatted:
