From robert@fledge.watson.org  Tue Aug 22 11:06:07 2000
Return-Path: <robert@fledge.watson.org>
Received: from fledge.watson.org (fledge.watson.org [204.156.12.50])
	by hub.freebsd.org (Postfix) with ESMTP id B7E8D37B422
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 22 Aug 2000 11:06:06 -0700 (PDT)
Received: (from robert@localhost)
	by fledge.watson.org (8.9.3/8.9.3) id OAA07374;
	Tue, 22 Aug 2000 14:06:06 -0400 (EDT)
	(envelope-from robert)
Message-Id: <200008221806.OAA07374@fledge.watson.org>
Date: Tue, 22 Aug 2000 14:06:06 -0400 (EDT)
From: rwatson@freebsd.org
Sender: robert@fledge.watson.org
Reply-To: rwatson@freebsd.org
To: FreeBSD-gnats-submit@freebsd.org
Subject: changing interface IP in same subnet doesn't propagate properly
X-Send-Pr-Version: 3.2

>Number:         20785
>Category:       kern
>Synopsis:       changing IP address on an interface may not propagate properly
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    ru
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Aug 22 11:10:01 PDT 2000
>Closed-Date:    Mon May 14 01:31:36 PDT 2001
>Last-Modified:  Mon May 14 01:32:39 PDT 2001
>Originator:     Robert Watson
>Release:        FreeBSD 4.1-STABLE i386
>Organization:
>Environment:

numocking# uname -a
FreeBSD numocking.gw.tislabs.com 4.1-STABLE FreeBSD 4.1-STABLE #1: Mon Aug  7 14
:13:19 EDT 2000     root@numocking.gw.tislabs.com:/usr/src/sys/compile/MOCKING
i386

>Description:

Changing the IP address of an interface may not necessarily cause new
sockets to make use of the new IP address, if the address is in the same
subnet as the previous address under IPv4.  This only seems to affect
outgoing packets to non-local addresses; the correct IP is used for outgoing
local packets on the subnet.

>How-To-Repeat:

(terminal one)
ifconfig xl0 inet 192.168.11.200
route add default 192.168.11.1

(terminal two)
tcpdump -eni xl0

(terminal one)
ifconfig xl0 inet 192.168.11.100
ping -c 100 128.2.1.2

(terminal two output)
tcpdump: listening on xl0
14:00:24.529628 0:c0:4f:3b:cb:62 0:a0:cc:67:f6:fe 0800 98: 192.168.11.200 > 128.
2.1.2: icmp: echo request
14:00:24.568225 0:a0:cc:67:f6:fe 0:c0:4f:3b:cb:62 0800 98: 128.2.1.2 > 192.168.1
1.200: icmp: echo reply

And needless to say, the echo responses don't propagate back up the stack
to the ping application, as they are for the wrong IP.  A ping of 192.168.11.1
works fine.

>Fix:

Work-around is to revert back to the old IP, or to delete and re-add the
default gateway route entry.  Presumably the outgoing IP address is tied
to the route entry somehow, so any use of that route forces use of the
old IP, which is no longer valid for that interface.  Given that the new IP
address is in the same subnet, ideally we could make use of the same route
entry.  In any case, continuing to make use of the now defunct IP address
is counter-intuitive (and misbehavior, in my view).  I have not tried this
with interfaces other than xl0.


>Release-Note:
>Audit-Trail:

From: Ruslan Ermilov <ru@FreeBSD.org>
To: Robert Watson <rwatson@FreeBSD.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/20785: changing interface IP in same subnet doesn't propagate properly
Date: Wed, 30 Aug 2000 18:00:07 +0300

 On Tue, Aug 22, 2000 at 02:06:06PM -0400, rwatson@FreeBSD.org wrote:
 > 
 > Changing the IP address of an interface may not necessarily cause new
 > sockets to make use of the new IP address, if the address is in the same
 > subnet as the previous address under IPv4.  This only seems to affect
 > outgoing packets to non-local addresses; the correct IP is used for outgoing
 > local packets on the subnet.
 > 
 I recently experienced this bug (with exact the same workaround) when I was
 playing with my libalias(3) ICMP fixes.  Yesterday, when I was playing with
 my kernel ICMP error generation fixes, I noticed that a similar bug happens
 with `-reject' static routes and thought that this might be related to this
 bug.
 
 1. Add the `-reject' route on your gateway machine:
 	# route add -host 1.2.3.4 -iface lo0 -reject
 
 2. From another box ping this host:
 	$ ping -c1 1.2.3.4
    You will not see the ICMP unreachable message.
 
 3. On the router box, re-add the default route:
 	# route delete default
 	# route add default $foo
 
 4. From another box try to ping this host again:
 	$ ping -c1 1.2.3.4
    Now you will see the ICMP unreachable message.
 
 
 -- 
 Ruslan Ermilov		Oracle Developer/DBA,
 ru@sunbay.com		Sunbay Software AG,
 ru@FreeBSD.org		FreeBSD committer,
 +380.652.512.251	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
Responsible-Changed-From-To: freebsd-bugs->ru 
Responsible-Changed-By: ru 
Responsible-Changed-When: Wed Oct 11 09:28:36 PDT 2000 
Responsible-Changed-Why:  
I am working on this issue. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=20785 

From: Ruslan Ermilov <ru@FreeBSD.org>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/20785: changing interface IP in same subnet doesn't propagate properly
Date: Wed, 11 Oct 2000 19:21:04 +0300

 --YiEDa0DAkWCtVeE4
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 As we now have a second complain (PR 21914), I feel I better let
 the others know that I was working on resolving this issue.
 
 Attached is the latest message from my private discussion with
 Garrett on this topic.
 
 
 -- 
 Ruslan Ermilov		Oracle Developer/DBA,
 ru@sunbay.com		Sunbay Software AG,
 ru@FreeBSD.org		FreeBSD committer,
 +380.652.512.251	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
 --YiEDa0DAkWCtVeE4
 Content-Type: message/rfc822
 Content-Disposition: inline
 
 Return-Path: <wollman@khavrinen.lcs.mit.edu>
 Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
 	by whale.sunbay.crimea.ua (8.11.0/8.11.0) with ESMTP id e93JU0I68961
 	for <ru@sunbay.crimea.ua>; Tue, 3 Oct 2000 22:30:02 +0300 (EEST)
 	(envelope-from wollman@khavrinen.lcs.mit.edu)
 Received: from hub.freebsd.org (hub.FreeBSD.org [216.136.204.18])
 	by mx1.FreeBSD.org (Postfix) with ESMTP id AF79A6E2E77
 	for <ru@sunbay.crimea.ua>; Tue,  3 Oct 2000 12:03:12 -0700 (PDT)
 Received: by hub.freebsd.org (Postfix)
 	id ABF1F37B66E; Tue,  3 Oct 2000 12:03:12 -0700 (PDT)
 Delivered-To: ru@freebsd.org
 Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193])
 	by hub.freebsd.org (Postfix) with ESMTP id 122B237B503
 	for <ru@FreeBSD.org>; Tue,  3 Oct 2000 12:03:12 -0700 (PDT)
 Received: (from wollman@localhost)
 	by khavrinen.lcs.mit.edu (8.9.3/8.9.3) id PAA43047;
 	Tue, 3 Oct 2000 15:03:10 -0400 (EDT)
 	(envelope-from wollman)
 Date: Tue, 3 Oct 2000 15:03:10 -0400 (EDT)
 From: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
 Message-Id: <200010031903.PAA43047@khavrinen.lcs.mit.edu>
 To: Ruslan Ermilov <ru@FreeBSD.org>
 Subject: Re: Please review: route.c patch
 In-Reply-To: <20000928184710.A93798@sunbay.com>
 References: <20000927200815.A59390@sunbay.com>
 	<200009271716.NAA73597@khavrinen.lcs.mit.edu>
 	<20000928131941.A84046@sunbay.com>
 	<20000928184710.A93798@sunbay.com>
 MIME-Version: 1.0
 
 <<On Thu, 28 Sep 2000 18:47:10 +0300, Ruslan Ermilov <ru@FreeBSD.org> said:
 
 > Say, for example, you have the ed0 network interface.
 
 > 1. You assing an IP address of 192.168.1.1 to it with ifconfig(8).
 > 2. You then add the `default' route to 192.168.1.100 with route(8).
 >    This will create the RTF_PRCLONING routing entry with rt_ifp pointing
 >    to the ed0 interface, and rt_ifa pointing to the 192.168.1.1 address.
 >    This is all OK so far.  Now the problem:
 > 3. You decide to delete the 192.168.1.1 address from the ed0 interface.
 >    This will delete the address from the interface's address list and
 >    from the in_ifaddrhead list, but WILL NOT invalidate existing routing
 >    entries that have their rt_ifa pointing to 192.168.1.1 address.
 
 It is supposed to.  I guess that's what we're discussing here (my
 context on this discussion is partially swapped out).
 
 > The solution: when we delete the `afamily' address from network interface,
 > we should also invalidate (delete?) all `afamily' routes that have their
 > rt_ifp -> to that interface and rt_ifa -> to that address.
 
 I never really found a satisfactory solution which makes me totally
 comfortable with this approach.  My concern is that the routes may
 represent important state which should (in certain circumstances) be
 changed rather than dropped completely, and that this should happen
 under the control of a routing process if there is one.  OTOH, I think
 what you are proposing would *work*, so perhaps it's the best solution
 for now.  (Most of the routing code needs to be re-architected in any
 case.)
 
 -GAWollman
 
 
 --YiEDa0DAkWCtVeE4--
 
State-Changed-From-To: open->analyzed 
State-Changed-By: ru 
State-Changed-When: Thu Mar 15 06:53:54 PST 2001 
State-Changed-Why:  
The fix has been committed to 5.0-CURRENT: 

Revision  Changes    Path 
1.25      +9 -4      src/usr.bin/netstat/netstat.1 
1.47      +3 -3      src/usr.bin/netstat/route.c 
1.61      +5 -4      src/sys/net/route.c 
1.51      +12 -1     src/sys/netinet/in.c 
1.38      +9 -11     src/sys/netinet/in_rmx.c 

http://www.freebsd.org/cgi/query-pr.cgi?pr=20785 
State-Changed-From-To: analyzed->closed 
State-Changed-By: ru 
State-Changed-When: Mon May 14 01:31:36 PDT 2001 
State-Changed-Why:  
The fix (slightly evolved) has been MFCed into 4.3-STABLE. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=20785 
>Unformatted:
