From nobody  Mon Mar  3 09:32:31 1997
Received: (from nobody@localhost)
          by freefall.freebsd.org (8.8.5/8.8.5) id JAA15264;
          Mon, 3 Mar 1997 09:32:31 -0800 (PST)
Message-Id: <199703031732.JAA15264@freefall.freebsd.org>
Date: Mon, 3 Mar 1997 09:32:31 -0800 (PST)
From: gilham@csl.sri.com
To: freebsd-gnats-submit@freebsd.org
Subject: FreeBSD NFS client can't mount filesystem from dual-homed machine
X-Send-Pr-Version: www-1.0

>Number:         2858
>Category:       kern
>Synopsis:       FreeBSD NFS client can't mount filesystem from dual-homed machine
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    peter
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar  3 09:40:01 PST 1997
>Closed-Date:    Tue Mar 27 04:30:28 PST 2001
>Last-Modified:  Tue Mar 27 05:23:07 PST 2001
>Originator:     Fred Gilham
>Release:        2.2-GAMMA
>Organization:
SRI International
>Environment:
FreeBSD japonica.csl.sri.com 2.2-970205-GAMMA FreeBSD 2.2-970205-GAMMA #4: Mon Mar  3 08:10:42 PST 1997     gilham@japonica.csl.sri.com:/usr/src/sys/compile/JAPONICA  i386

>Description:
If a FreeBSD NFS client tries to mount a filesystem from a multi-homed
NFS server, if it uses the `remoter' of the two addresses it will hang.
That's because the server will reply from the nearer of the two addresses
and the client will return an ICMP port-unreachable error.

You may be able to ^C out of the attempted mount but eventually the
machine will become unresponsive.

This occurs in a slightly different form in 2.1.7.  The mount seems
to succeed but you can't access the remote directory.  I'm waiting to
see if the 2.1.7 client dies. :-)

This DOES NOT happen with a Solaris 2 or SunOS 4 client.
>How-To-Repeat:
From FreeBSD NFS client, try to mount FS from multi-homed
server:

mount <IP of nearer interface>:/<filesystem> /mnt

This will succeed.  Unmount it, then

mount <IP of remoter interface>:/<filesystem> /mnt

This will hang and eventually your machine will hang.  (You may be
able to ^C out of the mount, but eventually the machine will stop
responding.)

>Fix:
Not known.  There was a patch I got some time ago from Terry Lambert that
was supposed to defeat the anti-spoofing code in /sys/nfs/nfs_subr.c but
it didn't help.

>Release-Note:
>Audit-Trail:

From: Fred Gilham <gilham@csl.sri.com>
To: freebsd-gnats-submit@freebsd.org, gilham@csl.sri.com
Cc:  Subject: Re: kern/2858: FreeBSD NFS client can't mount filesystem from dual-homed machine 
Date: Wed, 12 Mar 1997 11:01:50 -0800 (PST)

 I've found that the problem doesn't occur if you use TCP-NFS instead
 of UDP.  This will provide me with a workaround for the time being.
 
 -Fred Gilham    gilham@csl.sri.com
Responsible-Changed-From-To: freebsd-bugs->dfr 
Responsible-Changed-By: dfr 
Responsible-Changed-When: Sat Apr 5 02:01:19 PST 1997 
Responsible-Changed-Why:  
I'm going to fix this. 

From: Tor Egge <Tor.Egge@idi.ntnu.no>
To: freebsd-gnats-submit@freebsd.org, gilham@csl.sri.com, dfr@freebsd.org
Cc:  Subject: Re: kern/2858: FreeBSD NFS client can't mount filesystem from dual-homed machine
Date: Fri, 29 Aug 1997 00:57:55 +0200

 A different workaround is to use the `-c' option in order to not
 connect the udp socket to the wrong address.
 
 I recently experienced the same problem. I did not know that the
 machine was multi-homed (it has different host names on the different
 interfaces, and each host name only maps to one interface). Not using
 the `-c' caused mount_nfs to hang in the VOP_GETATTR call. Pressing ^C
 brought me back to the shell prompt, but this only meant that the
 `mount' program was aborted. The mount_nfs program was still hanging
 while holding a lock on `/mnt'. df did not show `/mnt' as being
 mounted, and I retried the mount command, causing the `mount' program
 to wait for a lock on `/mnt' while holding a lock on '/'.
 
 A missing option to a normal command should not have such a
 devastating effect, and I've made a tentative patch to the mount_nfs
 program that ought to reduce the problem. It assumes that the
 multi-homed NFS server uses a static routing table. If gethostbyname()
 returns multiple addresses, the address used in the answer from the
 server is selected. 
 
 2.2-STABLE patch:
 Index: mount_nfs.c
 ===================================================================
 RCS file: /home/ncvs/src/sbin/mount_nfs/mount_nfs.c,v
 retrieving revision 1.14.2.2
 diff -u -r1.14.2.2 mount_nfs.c
 --- mount_nfs.c	1997/05/14 12:06:34	1.14.2.2
 +++ mount_nfs.c	1997/08/28 19:34:36
 @@ -580,10 +580,12 @@
   * Return RPC_SUCCESS if server responds.
   */
  enum clnt_stat
 -pingnfsserver(addr, version, sotype)
 +pingnfsserver(addr, version, sotype, hp, connected)
  	struct sockaddr_in *addr;
  	int version;
  	int sotype;
 +	struct hostent *hp;
 +	int connected;
  {
  	struct sockaddr_in sin;
  	int tport;
 @@ -601,6 +603,169 @@
  
  	sin.sin_port = htons(tport);
  
 +	/*
 +	 * We need to verify that we get answers from the correct IP
 +	 * address. If not, we are very quickly subject to first a hanging
 +	 * mount_nfs process, then a hanging system.
 +	 */
 +	if (connected && sotype == SOCK_DGRAM) {
 +		fd_set readfds;
 +		struct sockaddr_in laddr;
 +		struct sockaddr_in from;
 +		int fromlen;
 +		int msglen;
 +		XDR xdrs;
 +		char buf[UDPMSGSIZE];
 +		struct rpc_msg msg;
 +		AUTH *auth;
 +		struct timeval now;
 +		unsigned long xid;
 +		struct rpc_err error;
 +		char **addrp;
 +		int recurse;
 +
 +		recurse = 0;
 +		
 +		auth = authunix_create_default();
 +		
 +		(void)gettimeofday(&now, (struct timezone *)0);
 +		xid = msg.rm_xid = getpid() ^ now.tv_sec ^ now.tv_usec;
 +		msg.rm_direction = CALL;
 +		msg.rm_call.cb_prog = RPCPROG_NFS;
 +		msg.rm_call.cb_vers = version;
 +		msg.rm_call.cb_proc = NFSPROC_NULL;
 +		msg.rm_call.cb_rpcvers = RPC_MSG_VERSION;
 +		msg.rm_call.cb_cred = auth->ah_cred;
 +		msg.rm_call.cb_verf = auth->ah_verf;
 +		
 +		xdrmem_create(&xdrs,buf,sizeof(buf),XDR_ENCODE);
 +		
 +		try.tv_sec = 0;
 +		try.tv_usec = 0;
 +		
 +		if (!xdr_callmsg(&xdrs,&msg)) {
 +			XDR_DESTROY(&xdrs);
 +			AUTH_DESTROY(auth);
 +			return RPC_CANTENCODEARGS;
 +		}
 +		msglen = (int) XDR_GETPOS(&xdrs);
 +		XDR_DESTROY(&xdrs);
 +		AUTH_DESTROY(auth);
 +		
 +		so = socket(AF_INET,SOCK_DGRAM,IPPROTO_UDP);
 +		if (so<0) 
 +			return RPC_CANTSEND;
 +
 +		try.tv_sec = 10;
 +		try.tv_usec = 0;
 +		
 +		setsockopt(so, SOL_SOCKET, SO_RCVTIMEO, &try,sizeof(try));
 +		
 +		memset(&laddr,0,sizeof(laddr));
 +		laddr.sin_family = AF_INET;
 +		laddr.sin_len = sizeof(laddr);
 +		
 +		bindresvport(so,&laddr);
 +		
 +		if (!hp || !hp->h_addr_list[1]) {
 +		  if (connect(so,(struct sockaddr *) &sin,sizeof(sin))<0) {
 +		    close(so);
 +		    return RPC_CANTSEND;
 +		  }
 +
 +		  if (send(so,buf,msglen,0)!=msglen) {
 +		    close(so);
 +		    return RPC_CANTSEND;
 +		  }
 +		  
 +		  msglen = recv(so,buf,sizeof(buf),0);
 +		  close(so);
 +		} else {
 +			if (sendto(so,buf,msglen,0,
 +				   (struct sockaddr *) &sin,
 +				   sizeof(sin))!=msglen) {
 +				close(so);
 +				return RPC_CANTSEND;
 +			}
 +			
 +			fromlen = sizeof(from);
 +			msglen = recvfrom(so,buf,sizeof(buf),0,
 +					  (struct sockaddr *) &from,
 +					  &fromlen);
 +			close(so);
 +			if (msglen>=0) {
 +				
 +				/* Ensure that answer is from correct host */
 +				
 +				for (addrp = hp->h_addr_list; *addrp;addrp++) 
 +					if (!memcmp(*addrp,&from.sin_addr,
 +						    sizeof(from.sin_addr)))
 +						break;
 +				if (!*addrp)
 +					return RPC_CANTRECV;
 +				
 +				/* Ensure that answer is from correct port */
 +				
 +				if (from.sin_port != sin.sin_port)
 +					return RPC_CANTRECV;
 +				
 +				/* 
 +				 * Replace IP address we will use for
 +				 * NFS access 
 +				 */
 +				
 +				if (memcmp(&addr->sin_addr,*addrp,
 +					   sizeof(addr->sin_addr))) {
 +				  memcpy(&addr->sin_addr,
 +					 *addrp,
 +					 sizeof(addr->sin_addr));
 +				  recurse=1;
 +				}
 +			}
 +		}
 +		
 +		if (msglen<0) {
 +			if (errno == EAGAIN)
 +				return RPC_TIMEDOUT;
 +			if (errno == ECONNREFUSED)
 +				return RPC_PROGUNAVAIL;
 +			return RPC_CANTRECV;
 +		}
 +		
 +		if (msglen<(int) sizeof(u_long))
 +			return RPC_CANTDECODEARGS;
 +		
 +		xdrmem_create(&xdrs,buf,msglen,XDR_DECODE);
 +		
 +		if (!xdr_u_long(&xdrs,&msg.rm_xid) ||
 +		    !xdr_enum(&xdrs,(enum_t *) &msg.rm_direction) ||
 +		    msg.rm_direction != REPLY ||
 +		    msg.rm_xid != xid) {
 +			XDR_DESTROY(&xdrs);
 +			return RPC_CANTDECODEARGS;
 +		}
 +		
 +		msg.acpted_rply.ar_verf = _null_auth;
 +		msg.acpted_rply.ar_results.where = NULL;
 +		msg.acpted_rply.ar_results.proc = xdr_void;
 +
 +		XDR_SETPOS(&xdrs,0);
 +		
 +		if (xdr_replymsg(&xdrs,&msg)) {
 +			_seterr_reply(&msg,&error);
 +			XDR_DESTROY(&xdrs);
 +			if (error.re_status == RPC_SUCCESS && recurse)
 +				return pingnfsserver(addr,
 +						     version,
 +						     sotype,
 +						     NULL,
 +						     connected);
 +			return error.re_status;
 +		}
 +		XDR_DESTROY(&xdrs);
 +		return RPC_CANTDECODEARGS;
 +	}
 +
  	pertry.tv_sec = 10;
  	pertry.tv_usec = 0;
  	if (sotype == SOCK_STREAM)
 @@ -693,6 +858,7 @@
  	 * Handle an internet host address and reverse resolve it if
  	 * doing Kerberos.
  	 */
 +	hp = NULL;
  	if (isdigit(*hostp)) {
  		if ((saddr.sin_addr.s_addr = inet_addr(hostp)) == -1) {
  			warnx("bad net address %s", hostp);
 @@ -731,12 +897,25 @@
  		nfsargsp->flags &= ~NFSMNT_NFSV3;
  	}
  	nfhret.stat = EACCES;	/* Mark not yet successful */
 -			/*
 -			 * First ping the nfs server to see if it supports
 -			 * the version of the protocol we want to use.
 -			 */
 -			clnt_stat = pingnfsserver(&saddr, nfsvers,
 -						  nfsargsp->sotype);
 +	while (retrycnt > 0) {
 +		saddr.sin_port = htons(PMAPPORT);
 +		saddr.sin_family = AF_INET;
 +		/*
 +		 * First ping the nfs server to see if it supports
 +		 * the version of the protocol we want to use.
 +		 */
 +		if ((tport = port_no ? port_no :
 +		     pmap_getport(&saddr, RPCPROG_NFS,
 +		    		  nfsvers, nfsproto)) == 0) {
 +			if ((opflags & ISBGRND) == 0)
 +				clnt_pcreateerror("NFS Portmap");
 +		} else 	if ((clnt_stat = 
 +			     pingnfsserver(&saddr, nfsvers,
 +					   nfsargsp->sotype,
 +					   hp,
 +					   !(nfsargsp->flags & 
 +					     NFSMNT_NOCONN)))!=
 +			    RPC_SUCCESS) {
  			if (clnt_stat == RPC_PROGVERSMISMATCH) {
  				if (mountmode == ANY) {
  					mountmode = V2;
 @@ -744,15 +923,10 @@
  				} else {
  					errx(1, "Can't contact NFS server");
  				}
 +			} else {
 +				warnx("Can't contact NFS server: %s",
 +				      clnt_sperrno(clnt_stat));
  			}
 -	while (retrycnt > 0) {
 -		saddr.sin_family = AF_INET;
 -		saddr.sin_port = htons(PMAPPORT);
 -		if ((tport = port_no ? port_no :
 -		     pmap_getport(&saddr, RPCPROG_NFS,
 -		    		  nfsvers, nfsproto)) == 0) {
 -			if ((opflags & ISBGRND) == 0)
 -				clnt_pcreateerror("NFS Portmap");
  		} else {
  			saddr.sin_port = 0;
  			pertry.tv_sec = 10;
 
 
 3.0-CURRENT patch:
 Index: mount_nfs.c
 ===================================================================
 RCS file: /home/ncvs/src/sbin/mount_nfs/mount_nfs.c,v
 retrieving revision 1.23
 diff -u -r1.23 mount_nfs.c
 --- mount_nfs.c	1997/06/03 13:49:26	1.23
 +++ mount_nfs.c	1997/08/28 19:44:47
 @@ -573,10 +573,12 @@
   * Return RPC_SUCCESS if server responds.
   */
  enum clnt_stat
 -pingnfsserver(addr, version, sotype)
 +pingnfsserver(addr, version, sotype, hp, connected)
  	struct sockaddr_in *addr;
  	int version;
  	int sotype;
 +	struct hostent *hp;
 +	int connected;
  {
  	struct sockaddr_in sin;
  	int tport;
 @@ -594,6 +596,169 @@
  
  	sin.sin_port = htons(tport);
  
 +	/*
 +	 * We need to verify that we get answers from the correct IP
 +	 * address. If not, we are very quickly subject to first a hanging
 +	 * mount_nfs process, then a hanging system.
 +	 */
 +	if (connected && sotype == SOCK_DGRAM) {
 +		fd_set readfds;
 +		struct sockaddr_in laddr;
 +		struct sockaddr_in from;
 +		int fromlen;
 +		int msglen;
 +		XDR xdrs;
 +		char buf[UDPMSGSIZE];
 +		struct rpc_msg msg;
 +		AUTH *auth;
 +		struct timeval now;
 +		u_int32_t xid;
 +		struct rpc_err error;
 +		char **addrp;
 +		int recurse;
 +		
 +		recurse = 0;
 +
 +		auth = authunix_create_default();
 +		
 +		(void)gettimeofday(&now, (struct timezone *)0);
 +		xid = msg.rm_xid = getpid() ^ now.tv_sec ^ now.tv_usec;
 +		msg.rm_direction = CALL;
 +		msg.rm_call.cb_prog = RPCPROG_NFS;
 +		msg.rm_call.cb_vers = version;
 +		msg.rm_call.cb_proc = NFSPROC_NULL;
 +		msg.rm_call.cb_rpcvers = RPC_MSG_VERSION;
 +		msg.rm_call.cb_cred = auth->ah_cred;
 +		msg.rm_call.cb_verf = auth->ah_verf;
 +		
 +		xdrmem_create(&xdrs,buf,sizeof(buf),XDR_ENCODE);
 +		
 +		try.tv_sec = 0;
 +		try.tv_usec = 0;
 +		
 +		if (!xdr_callmsg(&xdrs,&msg)) {
 +			XDR_DESTROY(&xdrs);
 +			AUTH_DESTROY(auth);
 +			return RPC_CANTENCODEARGS;
 +		}
 +		msglen = (int) XDR_GETPOS(&xdrs);
 +		XDR_DESTROY(&xdrs);
 +		AUTH_DESTROY(auth);
 +		
 +		so = socket(AF_INET,SOCK_DGRAM,IPPROTO_UDP);
 +		if (so<0) 
 +			return RPC_CANTSEND;
 +
 +		try.tv_sec = 10;
 +		try.tv_usec = 0;
 +		
 +		setsockopt(so, SOL_SOCKET, SO_RCVTIMEO, &try,sizeof(try));
 +		
 +		memset(&laddr,0,sizeof(laddr));
 +		laddr.sin_family = AF_INET;
 +		laddr.sin_len = sizeof(laddr);
 +		
 +		bindresvport(so,&laddr);
 +		
 +		if (!hp || !hp->h_addr_list[1]) {
 +		  if (connect(so,(struct sockaddr *) &sin,sizeof(sin))<0) {
 +		    close(so);
 +		    return RPC_CANTSEND;
 +		  }
 +
 +		  if (send(so,buf,msglen,0)!=msglen) {
 +		    close(so);
 +		    return RPC_CANTSEND;
 +		  }
 +		  
 +		  msglen = recv(so,buf,sizeof(buf),0);
 +		  close(so);
 +		} else {
 +			if (sendto(so,buf,msglen,0,
 +				   (struct sockaddr *) &sin,
 +				   sizeof(sin))!=msglen) {
 +				close(so);
 +				return RPC_CANTSEND;
 +			}
 +			
 +			fromlen = sizeof(from);
 +			msglen = recvfrom(so,buf,sizeof(buf),0,
 +					  (struct sockaddr *) &from,
 +					  &fromlen);
 +			close(so);
 +			if (msglen>=0) {
 +				
 +				/* Ensure that answer is from correct host */
 +				
 +				for (addrp = hp->h_addr_list; *addrp;addrp++) 
 +					if (!memcmp(*addrp,&from.sin_addr,
 +						    sizeof(from.sin_addr)))
 +						break;
 +				if (!*addrp)
 +					return RPC_CANTRECV;
 +				
 +				/* Ensure that answer is from correct port */
 +				
 +				if (from.sin_port != sin.sin_port)
 +					return RPC_CANTRECV;
 +				
 +				/* 
 +				 * Replace IP address we will use for
 +				 * NFS access 
 +				 */
 +				
 +				if (memcmp(&addr->sin_addr,*addrp,
 +					   sizeof(addr->sin_addr))) {
 +				  memcpy(&addr->sin_addr,
 +					 *addrp,
 +					 sizeof(addr->sin_addr));
 +				  recurse=1;
 +				}
 +			}
 +		}
 +		
 +		if (msglen<0) {
 +			if (errno == EAGAIN)
 +				return RPC_TIMEDOUT;
 +			if (errno == ECONNREFUSED)
 +				return RPC_PROGUNAVAIL;
 +			return RPC_CANTRECV;
 +		}
 +		
 +		if (msglen<(int) sizeof(u_long))
 +			return RPC_CANTDECODEARGS;
 +		
 +		xdrmem_create(&xdrs,buf,msglen,XDR_DECODE);
 +		
 +		if (!xdr_u_int32_t(&xdrs,&msg.rm_xid) ||
 +		    !xdr_enum(&xdrs,(enum_t *) &msg.rm_direction) ||
 +		    msg.rm_direction != REPLY ||
 +		    msg.rm_xid != xid) {
 +			XDR_DESTROY(&xdrs);
 +			return RPC_CANTDECODEARGS;
 +		}
 +		
 +		msg.acpted_rply.ar_verf = _null_auth;
 +		msg.acpted_rply.ar_results.where = NULL;
 +		msg.acpted_rply.ar_results.proc = xdr_void;
 +
 +		XDR_SETPOS(&xdrs,0);
 +		
 +		if (xdr_replymsg(&xdrs,&msg)) {
 +			_seterr_reply(&msg,&error);
 +			XDR_DESTROY(&xdrs);
 +			if (error.re_status == RPC_SUCCESS && recurse)
 +				return pingnfsserver(addr,
 +						     version,
 +						     sotype,
 +						     NULL,
 +						     connected);
 +			return error.re_status;
 +		}
 +		XDR_DESTROY(&xdrs);
 +		return RPC_CANTDECODEARGS;
 +	}
 +
  	pertry.tv_sec = 10;
  	pertry.tv_usec = 0;
  	if (sotype == SOCK_STREAM)
 @@ -685,6 +850,7 @@
  	 * Handle an internet host address and reverse resolve it if
  	 * doing Kerberos.
  	 */
 +	hp = NULL;
  	if (isdigit(*hostp)) {
  		if ((saddr.sin_addr.s_addr = inet_addr(hostp)) == -1) {
  			warnx("bad net address %s", hostp);
 @@ -726,18 +892,22 @@
  	while (retrycnt > 0) {
  		saddr.sin_family = AF_INET;
  		saddr.sin_port = htons(PMAPPORT);
 +		/*
 +		 * First ping the nfs server to see if it supports
 +		 * the version of the protocol we want to use.
 +		 */
  		if ((tport = port_no ? port_no :
  		     pmap_getport(&saddr, RPCPROG_NFS,
  		    		  nfsvers, nfsproto)) == 0) {
  			if ((opflags & ISBGRND) == 0)
  				clnt_pcreateerror("NFS Portmap");
 -		} else {
 -			/*
 -			 * First ping the nfs server to see if it supports
 -			 * the version of the protocol we want to use.
 -			 */
 -			clnt_stat = pingnfsserver(&saddr, nfsvers,
 -						  nfsargsp->sotype);
 +		} else 	if ((clnt_stat = 
 +			     pingnfsserver(&saddr, nfsvers,
 +					   nfsargsp->sotype,
 +					   hp,
 +					   !(nfsargsp->flags & 
 +					     NFSMNT_NOCONN)))!=
 +			    RPC_SUCCESS) {
  			if (clnt_stat == RPC_PROGVERSMISMATCH) {
  				if (mountmode == ANY) {
  					mountmode = V2;
 @@ -745,7 +915,11 @@
  				} else {
  					errx(1, "Can't contact NFS server");
  				}
 +			} else {
 +				warnx("Can't contact NFS server: %s",
 +				      clnt_sperrno(clnt_stat));
  			}
 +		} else {
  			saddr.sin_port = 0;
  			pertry.tv_sec = 10;
  			pertry.tv_usec = 0;
 
 
 
 
 - Tor Egge
Responsible-Changed-From-To: dfr->peter 
Responsible-Changed-By: peter 
Responsible-Changed-When: Sun Apr 26 00:37:33 PDT 1998 
Responsible-Changed-Why:  
I'll take a look at this.. 
State-Changed-From-To: open->closed 
State-Changed-By: iedowse 
State-Changed-When: Tue Mar 27 04:30:28 PST 2001 
State-Changed-Why:  
This is more of an issue with the NFS server than the client; the 
server should reply using an address corresponding to the destination 
IP in the requests.  FreeBSD's nfsd now has the `-h' option, which 
can be used on a multi-homed FreeBSD server to work around the 
issue of incorrect source IPs. 

It appears that the pingnfsserver() function mentioned in this PR 
was added in revision 1.22 of mount_nfs.c; that should address the 
client-side issues. Whoops, actually it looks like pingnfsserver() 
was accidentally? removed in the recent TI-RPC commit to current. 
I'll pester Martin and Alfred about that and re-open this PR if 
it isn't fixed quickly... 

http://www.freebsd.org/cgi/query-pr.cgi?pr=2858 
>Unformatted:
