From nobody@FreeBSD.org  Sat Sep 29 18:11:20 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 7F17E16A41B
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 29 Sep 2007 18:11:20 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 6581213C468
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 29 Sep 2007 18:11:20 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.1/8.14.1) with ESMTP id l8TIBK0n016884
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 29 Sep 2007 18:11:20 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.1/8.14.1/Submit) id l8TIBKtP016883;
	Sat, 29 Sep 2007 18:11:20 GMT
	(envelope-from nobody)
Message-Id: <200709291811.l8TIBKtP016883@www.freebsd.org>
Date: Sat, 29 Sep 2007 18:11:20 GMT
From: Michael Conlen <meconlen@obfuscated.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: NFS over UDP with kern.ipc.nmbclusters=0 does not work
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         116748
>Category:       kern
>Synopsis:       [nfs] NFS over UDP with kern.ipc.nmbclusters=0 does not work
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Sep 29 18:20:03 GMT 2007
>Closed-Date:    Fri Nov 16 21:06:49 UTC 2007
>Last-Modified:  Fri Nov 16 21:06:49 UTC 2007
>Originator:     Michael Conlen
>Release:        FreeBSD 6.2-p7
>Organization:
>Environment:
FreeBSD web6.tarhost.com 6.2-RELEASE-p7 FreeBSD 6.2-RELEASE-p7 #8: Fri Aug 10 00:35:21 EDT 2007     root@web6.tarhost.com:/usr/obj/usr/src/sys/WWW  i386

>Description:
When setting 

kern.ipc.nmbclusters=0

in /boot/loader.conf, which to my understanding means "unlimited", and attempting to mount a NFS file system (used version 3) using UDP the mount works however read operations fail. The traffic goes across the network, the server responds, the packets make it in to the client (as seen in tcpdump) but the NFS client code never returns the data to the user app. 

When using a TCP mount the NFS system operates as expected with two issues. 
1) regular nfs send error 32 reported on the server
2) regular impossible packet length from nfs server on the server

In order to use UDP I found that I needed to set

kern.ipc.nmbclusters=65535

I tested two otherwise identically configured machines against the same NFS server (also FreeBSD though somewhat older since non critical security patches have not been applied, it is, after all, the production NFS server for a web cluster.) and found this one configuration change to be the critical one. 

 
>How-To-Repeat:
Setup an NFS server
Setup an NFS client
add 

kern.ipc.nmbclusters=0

in the file /boot/loader.conf on the client. 

mount an NFS file system from the server on the client using options


rw,nfsv3

attempt to read the directories on the client.

>Fix:
workaround is to not use nmbclusters=0. Solution unknown. I can provide system in my environment for debugging with ssh and root access. 


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-i386->freebsd-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Oct 1 06:32:26 UTC 2007 
Responsible-Changed-Why:  
This does not sound i386-specific. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=116748 
State-Changed-From-To: open->closed 
State-Changed-By: kmacy 
State-Changed-When: Fri Nov 16 21:06:24 UTC 2007 
State-Changed-Why:  

kern.ipc.nmbclusters=0 means zero, not unlimited. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=116748 
>Unformatted:
