From thomas@cuivre.fr.eu.org  Tue Sep  7 18:44:25 2004
Return-Path: <thomas@cuivre.fr.eu.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id CC61B16A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  7 Sep 2004 18:44:25 +0000 (GMT)
Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [82.225.155.84])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 6200043D68
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  7 Sep 2004 18:44:25 +0000 (GMT)
	(envelope-from thomas@cuivre.fr.eu.org)
Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000)
	id 0912D2C3D5; Tue,  7 Sep 2004 20:44:24 +0200 (CEST)
Message-Id: <20040907184424.0912D2C3D5@melusine.cuivre.fr.eu.org>
Date: Tue,  7 Sep 2004 20:44:24 +0200 (CEST)
From: Thomas Quinot <thomas@cuivre.fr.eu.org>
Reply-To: Thomas Quinot <thomas@cuivre.fr.eu.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: route lookup does not skip interfaces marked down
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         71474
>Category:       kern
>Synopsis:       [route] route lookup does not skip interfaces marked down
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-net
>State:          analyzed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Sep 07 18:50:22 GMT 2004
>Closed-Date:    
>Last-Modified:  Mon Jun 07 17:57:10 UTC 2010
>Originator:     Thomas Quinot
>Release:        FreeBSD 5.2-CURRENT i386
>Organization:
>Environment:
System: FreeBSD malevil.act-europe.fr 5.2-CURRENT FreeBSD 5.2-CURRENT #1: Mon Aug  2 16:09:18 CEST 2004     root@alexandria.act-europe.fr:/usr/obj/usr/src/HEAD/sys/ADACORE  i386

	
>Description:
	When a route is set, the interface corresponding to the destination
	hop is looked up among all existing interfaces, not only those that
	are marked up, causing the wrong interface to be selected in some cases.
>How-To-Repeat:
	This issue can be demonstrated using gif interfaces:

	# ifconfig gif0 create
	# ifconfig gif0 192.168.0.254 192.168.0.1
	# ifconfig gif0 down
	# ifconfig gif1 create
	# ifconfig gif1 192.168.0.253 192.168.0.1

	Note at this point that 192.168.0.1 is reachable through interface
	gif1.

	# route add 192.168.1.0 192.168.0.1
	# netstat -rn | grep 192.168
	192.168.0.1        192.168.0.253      UH          1        0   gif1
	192.168.1          192.168.0.1        UGS         0        0   gif0

	Note then that, even though 192.168.0.1 is marked as reachable through
	gif1, for the purpose of routing to 192.168.1.0 the selected interface
	is gif0 (which is down).

>Fix:

	Fix not determined yet. This problem can be worked around by
	changing the addresses on the down interface so it won't clash
	with those of the up one:

	# ifconfig gif0 192.168.99.1 192.168.99.2
	# ifconfig gif0 down
	# route delete 192.168.1.0 192.168.0.1
	# route add 192.168.1.0 192.168.0.1
	# netstat -rn | grep 192.168
	192.168.0.1        192.168.0.253      UH          1        0   gif1
	192.168.1          192.168.0.1        UGS         0        0   gif1

>Release-Note:
>Audit-Trail:

From: Ruslan Ermilov <ru@FreeBSD.org>
To: Thomas Quinot <thomas@cuivre.fr.eu.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 8 Sep 2004 09:24:51 +0300

 --Ns7jmDPpOpCD+GE/
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable
 
 Just tell it the interface you really want:
 
 route add -net 192.168.1 -iface gif1
 
 
 Cheers,
 --=20
 Ruslan Ermilov
 ru@FreeBSD.org
 FreeBSD committer
 
 --Ns7jmDPpOpCD+GE/
 Content-Type: application/pgp-signature
 Content-Disposition: inline
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.6 (FreeBSD)
 
 iD8DBQFBPqWzqRfpzJluFF4RAiCTAJ0eXR3W+/fByp/PJJYkmUAOyzss3wCZAabP
 MZC2/HN4y8E4I5Ntz+KdMt0=
 =Amdw
 -----END PGP SIGNATURE-----
 
 --Ns7jmDPpOpCD+GE/--

From: Ruslan Ermilov <ru@freebsd.org>
To: Thomas Quinot <thomas@freebsd.org>
Cc: bug-followup@freebsd.org
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 8 Sep 2004 14:46:54 +0300

 On Wed, Sep 08, 2004 at 12:09:39PM +0200, Thomas Quinot wrote:
 > * Ruslan Ermilov, 2004-09-08 :
 > 
 > > This is *not* how I spelled it.  Try the above as shown.
 > 
 > I stand corrected. The command as proposed does work. Unfortunately this
 > applies only to point-to-point interfaces, not in the case where a
 > specific next-hop address is required.
 > 
 But your test case only applies to P2P interfaces, when both ends
 of two P2P interfaces share the same IP address.  I'd call this a
 misconfiguration.  I do not see how this could apply to broadcast
 interfaces.
 
 
 Cheers,
 -- 
 Ruslan Ermilov
 ru@FreeBSD.org
 FreeBSD committer

From: Ruslan Ermilov <ru@freebsd.org>
To: Thomas Quinot <thomas@freebsd.org>
Cc: bug-followup@freebsd.org
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 8 Sep 2004 15:49:26 +0300

 On Wed, Sep 08, 2004 at 02:26:29PM +0200, Thomas Quinot wrote:
 > * Ruslan Ermilov, 2004-09-08 :
 > 
 > >  But your test case only applies to P2P interfaces, when both ends
 > >  of two P2P interfaces share the same IP address.
 > 
 > No, this is not the case. Only the remote ends of the two interfaces
 > share the same IP address. I described that test case because using gif
 > interfaces makes it easier to reproduce, but I initially observed this
 > symptom with one bge and one Intel wireless (ndis) interface.
 > 
 That's what I meant (two remote ends of both interfaces).
 
 > >  I do not see how this could apply to broadcast interfaces.
 > 
 > As follows. Consider a machine with two Ethernet interfaces I1 and I2.
 > 
 > ifconfig I1 192.168.0.1 netmask 255.255.255.0
 > ifconfig I1 down
 > ifconfig I2 192.168.0.2 netmask 255.255.255.0
 > route add default 192.168.0.254
 > 
 > The default route will be set to 192.168.0.254 on I1, not on I2 as
 > expected.
 > 
 It's still a misconfiguration -- if you do not bring the I1 interface
 down, the "interface" route for I2 will not be installed, and the
 ifconfig(8) command will be aborted.  Anyway...
 
 I reproduced this with two ng_eiface(4) nodes.  There's still a useful
 option in the route(8) utility -- you can supply an explicit reference
 to the interface, like this:
 
 : route add default 192.168.0.254 -ifp I1
 
 This is also vaguely documented in the route(8) manpage, FWIW:
 
 : In a change or add command where the destination and gateway are not suf-
 : ficient to specify the route ..., the -ifp or -ifa modifiers may be used
 : to determine the interface or interface address.
 
 
 Cheers,
 -- 
 Ruslan Ermilov
 ru@FreeBSD.org
 FreeBSD committer

From: Ruslan Ermilov <ru@FreeBSD.ORG>
To: Thomas Quinot <thomas@FreeBSD.ORG>
Cc: bug-followup@FreeBSD.ORG
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 8 Sep 2004 16:49:27 +0300

 On Wed, Sep 08, 2004 at 03:06:48PM +0200, Thomas Quinot wrote:
 > * Ruslan Ermilov, 2004-09-08 :
 > 
 > >  > ifconfig I1 192.168.0.1 netmask 255.255.255.0
 > >  > ifconfig I1 down
 > >  > ifconfig I2 192.168.0.2 netmask 255.255.255.0
 > >  > route add default 192.168.0.254
 > >  > 
 > >  > The default route will be set to 192.168.0.254 on I1, not on I2 as
 > >  > expected.
 > >  > 
 > >  It's still a misconfiguration -- if you do not bring the I1 interface
 > >  down, the "interface" route for I2 will not be installed, and the
 > >  ifconfig(8) command will be aborted.  Anyway...
 > 
 > I am not sure I get what you mean. The point of this PR is that I *do*
 > bring I1 down, that the kernel has an unambiguous indication of how to
 > reach the specified gateway ('ping 192.168.0.254' works in the example
 > described above), and in spite of that the wrong interface is selected
 > when creating a route.
 > 
 This is a misconfiguration because you cannot expect two broadcast
 interfaces configured with the same IP network to work.  The fact
 that it works at all, in a scenario you shown, is probably a bug.
 
 > >  I reproduced this with two ng_eiface(4) nodes.  There's still a useful
 > >  option in the route(8) utility -- you can supply an explicit reference
 > >  to the interface, like this:
 > >  : route add default 192.168.0.254 -ifp I1
 > 
 > Right, I was not aware of that option.
 > 
 > >  This is also vaguely documented in the route(8) manpage, FWIW:
 > 
 > *Vaguely* is the right term I think. :-)
 > 
 > Having browsed through the code, I think the interface lookup for route
 > additions should use a modified version of ifa_ifwithaddr ignoring
 > non-up interfaces.
 > 
 If we change ifa_ifwith*() to ignore !IFF_UP interfaces, I suspect this
 will break too many things.  I think the correct change here would be
 to set the interface of the route to the interface of its gateway (if
 route is through the gateway (RTF_GATEWAY)).  This should be easy to
 fix, but I don't have a time for this now.
 
 
 Cheers,
 -- 
 Ruslan Ermilov
 ru@FreeBSD.org
 FreeBSD committer
Responsible-Changed-From-To: freebsd-bugs->bms 
Responsible-Changed-By: bms 
Responsible-Changed-When: Mon Sep 25 19:15:12 UTC 2006 
Responsible-Changed-Why:  
I'll take this 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71474 

From: Bruce M Simpson <bms@incunabulum.net>
To: freebsd-gnats-submit@FreeBSD.org
Cc: ru@FreeBSD.org,  glebius@FreeBSD.org,  andre@FreeBSD.org
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 27 Sep 2006 13:30:42 +0100

 Hi,
 
 I've been looking at this PR. A few points, based on code inspection and 
 reproducing the test case.
 
  *ifa_ifwithroute() already checks for RTF_GATEWAY, and unconditionally 
 calls ifa_ifwithdstaddr() to obtain the ifa pointer.
  * The rtmsg passed in from userland is well-formed (RTM_ADD).
  * ifa_ifwithroute() in this case is called from rt_getifa() during RTM_ADD.
  * ifa_ifwithdstaddr() is called from many other parts of the stack, so 
 being conservative about the change seems sensible.
  * This isn't really a multipath problem so much as a next-hop 
 resolution problem.
 
 Perhaps if we change ifa_ifwithdstaddr() to ifa_ifwithdstaddrflags() and 
 pass IFF_UP as a necessary flag for the interface match that would be an 
 acceptable fix, and change other consumers to pass 0 or use a macro to 
 emulate the old function.
 
 We have to call ifa_ifwithdstaddr() to look up the correct ifaddr for 
 the rt->rt_ifa anyway; the rt->rt_ifp can in turn be looked up from that.
 
 Just my 2c,
 BMS

From: Bruce M Simpson <bms@incunabulum.net>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 27 Sep 2006 13:59:20 +0100

 For a related patch, see also:
 http://lists.freebsd.org/pipermail/freebsd-net/2006-February/009884.html
 
 (Provided by glebius)
State-Changed-From-To: open->analyzed 
State-Changed-By: bms 
State-Changed-When: Wed Sep 27 13:25:46 UTC 2006 
State-Changed-Why:  
We can reproduce the problem, we've analyzed it, there are possible 
fixes, we need to decide what to do. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71474 

From: Bruce M Simpson <bms@incunabulum.net>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Mon, 19 Mar 2007 00:19:13 +0000

 This looks like an architectural problem stemming from how next-hop 
 resolution is performed in FreeBSD's forwarding plane. Fear not -- XORP 
 has similar problems; recursively resolving next-hops when they change 
 is not a trivial issue.
 
 To my mind this needs to be fixed with a rewrite. Looking at the code, 
 we'd have to do another tree walk on PRC_IFDOWN. We already do this, but 
 we ignore static routes.
 
 The reason why the default route doesn't get its ifa/ifp updated when 
 the interface it refers to goes down, is because it's static.
 
 

From: Bruce M Simpson <bms@incunabulum.net>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/71474: route lookup does not skip interfaces marked down
Date: Wed, 18 Jul 2007 00:01:56 +0100

 This sounds like a job for a 'floating static' forwarding entry.
 
 A 'floating static' entry is one whose next-hop may be re-resolved 
 although the entry itself is static and should be treated as such by the 
 forwarding code.
 
 A 'floating' flag would allow the PRC_IFDOWN path to tell the 
 difference, and update the ifp in those cases where the next-hop would 
 change, without clobbering the state of the next-hop for FIB entries 
 marked as static.
 
 See IOS or JunOS documentation for more information on 'floating static 
 routes'.
Responsible-Changed-From-To: bms->none 
Responsible-Changed-By: bms 
Responsible-Changed-When: Sun 6 Jun 2010 15:07:11 UTC 
Responsible-Changed-Why:  
I'm afraid I have no free time for the foreseeable future which 
I can commit directly to FreeBSD work, due to a demanding new role. 

I am happy to review the occasional patch, but regrettably cannot 
be directly involved in engineering work beyond this. 

Thank you for your understanding. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=71474 
Responsible-Changed-From-To: none->freebsd-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Jun 7 17:55:17 UTC 2010 
Responsible-Changed-Why:  
Canonicalize assignment. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71474 
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Jun 7 17:55:37 UTC 2010 
Responsible-Changed-Why:  
On second thought, it probably belongs to -net. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71474 
>Unformatted:
