From nobody@FreeBSD.org  Mon Aug  8 16:10:17 2005
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id C9ECF16A420
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  8 Aug 2005 16:10:17 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id E487843D5C
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  8 Aug 2005 16:10:06 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j78GA6QX072310
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 8 Aug 2005 16:10:06 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id j78GA6vo072309;
	Mon, 8 Aug 2005 16:10:06 GMT
	(envelope-from nobody)
Message-Id: <200508081610.j78GA6vo072309@www.freebsd.org>
Date: Mon, 8 Aug 2005 16:10:06 GMT
From: "Thom O'Connor" <thom@communigate.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: NFS client problem "Stale NFS file handle" 
X-Send-Pr-Version: www-2.3

>Number:         84673
>Category:       kern
>Synopsis:       [nfs] NFS client problem "Stale NFS file handle"
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Aug 08 16:20:13 GMT 2005
>Closed-Date:    Sun Jan 27 13:11:42 UTC 2008
>Last-Modified:  Sun Jan 27 13:11:42 UTC 2008
>Originator:     Thom O'Connor
>Release:        5.4-STABLE
>Organization:
CommuniGate Systems
>Environment:
FreeBSD sys8.cluster 5.4-STABLE FreeBSD 5.4-STABLE #0: Wed Aug  3 21:13:51 PDT 2005     root@sys8.test:/usr/obj/usr/src/sys/GENERIC  i386

FreeBSD sys7.cluster 5.4-RELEASE FreeBSD 5.4-RELEASE #0: Sun May  8 10:21:06 UTC 2005     root@harlow.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386

>Description:
Using FreeBSD as an NFS client and testing it against various NFS servers, it appears that FreeBSD is incorrectly handling file caching data to lead to situations where the error message "Stale NFS file handle" (errno.h error 70).

I've been able to repeat the problem using any of these operating systems:

FreeBSD 5.4-STABLE
FreeBSD 5.4-RELEASE
FreeBSD 4.6-RELEASE

I've tested against three different NFS servers, and the problem remained with each:

RAIDZONE NFS array (linux 2.2 kernel)
Linux 2.6 NFS server (SuSE 9.3)
FreeBSD 5.4 NFS server (5.4-RELEASE)

I've also tested on other operating systems, and none of these experienced this problem:

Solaris 8 SPARC
Linux 2.6 kernel (SuSE 9.3)
Linux 2.6 kernel (SuSE 9.1)

This seems to indicate the problem is with the FreeBSD NFS client code. Various NFS client mount options were tested, including tcp and mntudp, as well as setting nfs_access_cache="0" in /etc/rc.conf. None of these options affected the below behaviour.

Any time the underlying file (and inode, I suppose) is renamed, the "Stale NFS file handle" error will occur. I am not an NFS expert by any means, but our product uses files on an NFS filesystem in this manner, and we've never seen this problem on any other OS (CommuniGate Pro is available on 37 platforms today). The NFS client should not be caching file handle data referenced by a unique identifier other than the filename itself, from what I know.

Please contact me if you have any questions. Thank you.

Sincerely,

Thom O'Connor
thom@communigate.com
thom@interludium.com
>How-To-Repeat:
The problem is quite easy to replicate. Create an NFS server share, and mount that export on at least two systems, where at least one of the systems is a FreeBSD system.

In the following example, both sys7 is FreeBSD 5.4-RELEASE and sys8 is FreeBSD 5.4-STABLE:

Using /etc/fstab settings something like these (but varied for different tests):
raidzone.cluster:/home/data /data nfs rw,nfsv3,-r32768,-w32768,intr 0 0

And then mounting that filesystem on both sys7 and sys8:
sys7# mount /data
sys8# mount /data

Then perform these steps:

sys8# cd /data
sys8# touch testfile
sys8# cat testfile

sys7# cd /data
sys7# touch testfile2
sys7# mv testfile2 testfile

sys8# cat testfile
cat: testfile: Stale NFS file handle

>Fix:

>Release-Note:
>Audit-Trail:

From: Rong-En Fan <rafan@infor.org>
To: bug-followup@FreeBSD.org, thom@communigate.com
Cc:  
Subject: Re: kern/84673: [nfs] NFS client problem "Stale NFS file handle"
Date: Sat, 11 Mar 2006 10:56:10 +0800 (CST)

 Hi,
 
 I can not reproduce this problem on 6.1-PRERELEASE as of
 Feb 17. These two clients have identical kernel and world.
 Server is running 5.5-PRERELEASE as of Mar 10. The mount
 options are "rw,nosuid,bg,intr,nodev", rest nfs client
 related options are as default.
 
 Cheers,
 Rong-En Fan
Responsible-Changed-From-To: freebsd-bugs->cel 
Responsible-Changed-By: cel 
Responsible-Changed-When: Fri May 12 21:24:45 UTC 2006 
Responsible-Changed-Why:  
Though Rong-En Fan couldn't reproduce this with the provided instructions, 
I'm guessing this is an area that needs pretty thorough testing anyway, 
and we are bound to hit something. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=84673 
Responsible-Changed-From-To: cel->freebsd-bugs 
Responsible-Changed-By: cel 
Responsible-Changed-When: Mon Mar 12 15:33:38 UTC 2007 
Responsible-Changed-Why:  
Back to the public pool. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=84673 
State-Changed-From-To: open->feedback 
State-Changed-By: kmacy 
State-Changed-When: Fri Nov 16 01:45:08 UTC 2007 
State-Changed-Why:  

Likely stale. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=84673 
State-Changed-From-To: feedback->closed 
State-Changed-By: rwatson 
State-Changed-When: Sun Jan 27 13:10:26 UTC 2008 
State-Changed-Why:  
Closed due to feedback timeout (>2 months); it seems likely that this 
problem is resolved on recent FreeBSD versions.  If you have any further 
information that could be used to help debug this problem, please follow 
up on the PR by e-mail and we can re-open it.  Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=84673 
>Unformatted:
Have you seen this on RELENG_6? I've done this frequently without any issues.
