From nobody@FreeBSD.ORG  Thu Jan 20 15:19:18 2000
Return-Path: <nobody@FreeBSD.ORG>
Received: by hub.freebsd.org (Postfix, from userid 32767)
	id 1B499153A5; Thu, 20 Jan 2000 15:19:18 -0800 (PST)
Message-Id: <20000120231918.1B499153A5@hub.freebsd.org>
Date: Thu, 20 Jan 2000 15:19:18 -0800 (PST)
From: kannanv@research.bell-labs.com
Sender: nobody@FreeBSD.ORG
To: freebsd-gnats-submit@FreeBSD.org
Subject: ICMP error generation fails to correctly insert IP ID on returned packet
X-Send-Pr-Version: www-1.0

>Number:         16240
>Category:       kern
>Synopsis:       ICMP error generation fails to correctly insert IP ID on returned packet
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    ru
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jan 20 15:20:02 PST 2000
>Closed-Date:    Tue Sep 26 10:27:23 PDT 2000
>Last-Modified:  Tue Sep 26 10:28:31 PDT 2000
>Originator:     Kannan Varadhan
>Release:        3.3-RELEASE
>Organization:
Lucent Techn ologies, Bell Lans
>Environment:
FreeBSD yankees.mlb.research.bell-labs.com 3.3-RELEASE FreeBSD 3.3-RELEASE #0: Fri Oct 22 11:38:58 EDT 1999     root@dodgers.mlb.research.bell-labs.com:/usr/src/sys/compile/MH2A235  i386

>Description:
See tcpdump traces below:

16:31:54.084947 0:90:27:61:e7:99 0:50:4:b1:f0:90 0800 162: 135.104.73.82.2094666972 > 135.104.73.129.2049: 120 getattr [|nfs]
                         4500 0094 012d 0000 3d11 da88 8768 4952
                         8768 4981 03e5 0801 0080 d6a1 7cda 14dc 
                         0000 0000 0000 0002 0001 86a3 0000 0003
                         0000 0001 0000
### ORIGINATING PACKET.  NOTE IP ID is 0194
16:31:54.085012 0:50:4:b1:f0:90 0:60:1d:9:0:5a 0800 154: 135.104.73.11.2049 > 135.104.73.82.2094666972: reply ok 112 getattr [|nfs]
                         4500 008c 0194 0000 4011 d79f 8768 490b
                         8768 4952 0801 03e5 0078 152e 7cda 14dc
                         0000 0001 0000 0000 0000 0000 0000 0000
                         0000 0000 0000

### ICMP ERROR RETURNED.  NOTE IP ID on returned packet is 9401,
### indicating some byte-ordering problems.
16:31:54.085486 0:60:1d:9:0:5a 0:50:4:b1:f0:90 0800 70: 135.104.73.82 > 135.104.73.11: icmp: 135.104.73.82 udp port 997 unreachable
                         4500 0038 012e 0000 fd01 1b69 8768 4952
                         8768 490b 0303 5e31 0000 0000 4500 008c
                         9401 0000 3e11 d99f 8768 490b 8768 4952
                         0801 03e5 0078

>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:

From: Ruslan Ermilov <ru@FreeBSD.org>
To: kannanv@research.bell-labs.com,
	Garrett Wollman <wollman@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/16240: ICMP error generation fails to correctly insert IP ID on returned packet
Date: Fri, 21 Jan 2000 18:10:28 +0200

 --azLHFNyN32YCQGCU
 Content-Type: text/plain; charset=us-ascii
 
 On Thu, Jan 20, 2000 at 03:19:18PM -0800, kannanv@research.bell-labs.com wrote:
 > 
 > >Number:         16240
 > >Category:       kern
 > >Synopsis:       ICMP error generation fails to correctly insert IP ID on returned packet
 > >Severity:       non-critical
 [...]
 > See tcpdump traces below:
 > 
 > 16:31:54.084947 0:90:27:61:e7:99 0:50:4:b1:f0:90 0800 162: 135.104.73.82.2094666972 > 135.104.73.129.2049: 120 getattr [|nfs]
 >                          4500 0094 012d 0000 3d11 da88 8768 4952
 >                          8768 4981 03e5 0801 0080 d6a1 7cda 14dc 
 >                          0000 0000 0000 0002 0001 86a3 0000 0003
 >                          0000 0001 0000
 > ### ORIGINATING PACKET.  NOTE IP ID is 0194
 > 16:31:54.085012 0:50:4:b1:f0:90 0:60:1d:9:0:5a 0800 154: 135.104.73.11.2049 > 135.104.73.82.2094666972: reply ok 112 getattr [|nfs]
 >                          4500 008c 0194 0000 4011 d79f 8768 490b
 >                          8768 4952 0801 03e5 0078 152e 7cda 14dc
 >                          0000 0001 0000 0000 0000 0000 0000 0000
 >                          0000 0000 0000
 > 
 > ### ICMP ERROR RETURNED.  NOTE IP ID on returned packet is 9401,
 > ### indicating some byte-ordering problems.
 > 16:31:54.085486 0:60:1d:9:0:5a 0:50:4:b1:f0:90 0800 70: 135.104.73.82 > 135.104.73.11: icmp: 135.104.73.82 udp port 997 unreachable
 >                          4500 0038 012e 0000 fd01 1b69 8768 4952
 >                          8768 490b 0303 5e31 0000 0000 4500 008c
 >                          9401 0000 3e11 d99f 8768 490b 8768 4952
 >                          0801 03e5 0078
 > 
 It is an old known bug, for which there is no easy fix, see attached.
 
 -- 
 Ruslan Ermilov		Sysadmin and DBA of the
 ru@ucb.crimea.ua	United Commercial Bank,
 ru@FreeBSD.org		FreeBSD committer,
 +380.652.247.647	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: (from ru@localhost)
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) id MAA99165;
 	Thu, 11 Mar 1999 12:28:58 +0200 (EET)
 	(envelope-from ru)
 Date: Thu, 11 Mar 1999 12:28:58 +0200
 From: Ruslan Ermilov <ru@ucb.crimea.ua>
 To: wollman@freebsd.org, fenner@freebsd.org
 Cc: core@freebsd.org
 Subject: ip_input.c bug!!!
 Message-ID: <19990311122858.A77664@relay.ucb.crimea.ua>
 Mime-Version: 1.0
 Content-Type: text/plain; charset=us-ascii
 X-Mailer: Mutt 0.95.3i
 X-Operating-System: FreeBSD 3.1-STABLE i386
 
 Hi!
 
 I have discovered the bug in ip_input(), which makes ICMP
 to not work in some cases.
 
 This bug is only seen in FreeBSD 3.1-STABLE (RELENG_3), and,
 probably, in CURRENT.  RELENG_2_2 versions are not affected,
 I've tested this on 2.2.1-RELEASE and 2.2.8-RELEASE machines.
 
 The scenario
 ============
 When a router can't forward IP packet to another host when
 the host is down, it normally returns ICMP host unreachable.
 
 I noticed (with tcpdump) that router, running 3.1-STABLE
 doesn't return ICMP errors when I'm PINGing from either
 SCO OpenServer or SCO UnixWare7.
 
 Then I noticed with tcpdump that ICMP echo request packets
 from SCO's hosts have the DF bit set.
 
 
 The bug
 =======
 I've inserted the various debug printf's in both ip_forward()
 and icmp_error(), and here's what I've discovered.
 
 I'm talking about the following files/revisions:
 $Id: ip_input.c,v 1.111 1999/01/12 12:25:00 eivind Exp $
 $Id: ip_icmp.c,v 1.33 1998/12/04 04:21:25 dillon Exp $
 
 
 The ICMP echo request packet with DF bit set (ip_off=0x4000 in network
 byte order) is passed to the ip_input().
 
 1. ip_input():
    a) converts the `ip_len', `ip_id' and `ip_off' fields from network
    to host byte order (lines 343, 348 and 349);
    --> ip_off=0x4000 (in host byte order)
    b) performs the various validness tests on it;
    c) calls ip_forward(m,0) (line 553)
 
 2. ip_forward():
    a) saves at most 64 bytes of the original packet `m' into `mcopy',
    by means of m_copy() (line 1438);
    b) calls ip_output(m, ...)
    c) ip_output() changes the byte order of field `ip_off' (among others)
    to network representation
    --> mtod(m, struct ip *)->ip_off=0x0040
    --> mtod(mcopy, struct ip *)->ip_off=0x0040 also, because `m' and
    `mcopy' share the same data area.  I tested this by putting
    `printf' before and right after the call to `ip_output(m, ...)':
 
 +       printf("%x\n", mtod(mcopy, struct ip *)->ip_off);
         error = ip_output(m, (struct mbuf *)0, &ipforward_rt,
                           IP_FORWARDING, 0);
 +       printf("%x\n", mtod(mcopy, struct ip *)->ip_off);
         if (error)
                 ipstat.ips_cantforward++;
         else {
 
 
    d) ip_output() returns one of the following errors: ENETUNREACH,
    EHOSTUNREACH, ENETDOWN or EHOSTDOWN;
    e) ip_output() calls icmp_error(mcopy,ICMP_UNREACH,ICMP_UNREACH_HOST,...)
 
 3. icmp_error():
    a) checks whether this is the first fragment of message (line 137);
    --> and it fails because ip_off=0x0040
 
 
 The following patch solves the problem.
 It makes `m_copy' to contain the _real_ copy of the `m'.
 
 Index: ip_input.c
 ===================================================================
 RCS file: /usr/FreeBSD-CVS/src/sys/netinet/ip_input.c,v
 retrieving revision 1.111
 diff -u -r1.111 ip_input.c
 --- ip_input.c	1999/01/12 12:25:00	1.111
 +++ ip_input.c	1999/03/10 21:37:33
 @@ -1436,6 +1436,8 @@
  	 * we need to generate an ICMP message to the src.
  	 */
  	mcopy = m_copy(m, 0, imin((int)ip->ip_len, 64));
 +	if (mcopy && (mcopy->m_flags & M_EXT))
 +		mcopy = m_pullup(mcopy, mcopy->m_len);
  
  	/*
  	 * If forwarding packet using same interface that it came in on,
 
 
 
 
 Well, what I can't understand is why this works on 2.2.8 routers.
 It seems that something has been changed in src/sys/kern/uipc_mbuf.c
 since revision 1.26 which changed the way how the m_copym() works.
 
 Do you know _what_?
 
 
 
 Cheers,
 -- 
 Ruslan Ermilov		Sysadmin and DBA of the
 ru@ucb.crimea.ua	United Commercial Bank
 +380.652.247.647	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193])
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) with ESMTP id TAA99227
 	for <ru@ucb.crimea.ua>; Thu, 11 Mar 1999 19:18:24 +0200 (EET)
 	(envelope-from wollman@khavrinen.lcs.mit.edu)
 Received: (from wollman@localhost)
 	by khavrinen.lcs.mit.edu (8.9.1/8.9.1) id MAA01325;
 	Thu, 11 Mar 1999 12:17:22 -0500 (EST)
 	(envelope-from wollman)
 Date: Thu, 11 Mar 1999 12:17:22 -0500 (EST)
 From: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
 Message-Id: <199903111717.MAA01325@khavrinen.lcs.mit.edu>
 To: Ruslan Ermilov <ru@ucb.crimea.ua>
 Cc: wollman@FreeBSD.ORG, fenner@FreeBSD.ORG, core@FreeBSD.ORG
 Subject: ip_input.c bug!!!
 In-Reply-To: <19990311122858.A77664@relay.ucb.crimea.ua>
 References: <19990311122858.A77664@relay.ucb.crimea.ua>
 
 <<On Thu, 11 Mar 1999 12:28:58 +0200, Ruslan Ermilov <ru@ucb.crimea.ua> said:
 
 > Index: ip_input.c
 > ===================================================================
 > RCS file: /usr/FreeBSD-CVS/src/sys/netinet/ip_input.c,v
 > retrieving revision 1.111
 > diff -u -r1.111 ip_input.c
 > --- ip_input.c	1999/01/12 12:25:00	1.111
 > +++ ip_input.c	1999/03/10 21:37:33
 > @@ -1436,6 +1436,8 @@
 >  	 * we need to generate an ICMP message to the src.
 >  	 */
 >  	mcopy = m_copy(m, 0, imin((int)ip->ip_len, 64));
 > +	if (mcopy && (mcopy->m_flags & M_EXT))
 > +		mcopy = m_pullup(mcopy, mcopy->m_len);
  
 >  	/*
 >  	 * If forwarding packet using same interface that it came in on,
 
 This does look like it does as you suggest, but I don't think it's the
 right behavior -- we don't want to be hand-copying every packet as we
 send it.  The Right Thing is substantially more complicated; what we
 need to do is to change the interface to ip_output() (and by extension
 to if_output()) such that the original mbuf chain is not freed on
 error.  Then, ip_output would undo its manipulation of the packet (and
 expect the caller to free it after doing whatever was necessary).
 This would also speed up the general case of forwarding.
 
 -GAWollman
 
 --
 Garrett A. Wollman   | O Siem / We are all family / O Siem / We're all the same
 wollman@lcs.mit.edu  | O Siem / The fires of freedom 
 Opinions not those of| Dance in the burning flame
 MIT, LCS, CRS, or NSA|                     - Susan Aglukark and Chad Irschick
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: (from ru@localhost)
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) id TAA00814;
 	Thu, 11 Mar 1999 19:24:58 +0200 (EET)
 	(envelope-from ru)
 Date: Thu, 11 Mar 1999 19:24:57 +0200
 From: Ruslan Ermilov <ru@ucb.crimea.ua>
 To: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
 Cc: wollman@FreeBSD.ORG, fenner@FreeBSD.ORG, core@FreeBSD.ORG
 Subject: Re: ip_input.c bug!!!
 Message-ID: <19990311192457.A99244@relay.ucb.crimea.ua>
 References: <19990311122858.A77664@relay.ucb.crimea.ua> <199903111717.MAA01325@khavrinen.lcs.mit.edu>
 Mime-Version: 1.0
 Content-Type: text/plain; charset=us-ascii
 X-Mailer: Mutt 0.95.3i
 In-Reply-To: <199903111717.MAA01325@khavrinen.lcs.mit.edu>; from Garrett Wollman on Thu, Mar 11, 1999 at 12:17:22PM -0500
 X-Operating-System: FreeBSD 3.1-STABLE i386
 
 On Thu, Mar 11, 1999 at 12:17:22PM -0500, Garrett Wollman wrote:
 > <<On Thu, 11 Mar 1999 12:28:58 +0200, Ruslan Ermilov <ru@ucb.crimea.ua> said:
 > 
 > > Index: ip_input.c
 > > ===================================================================
 > > RCS file: /usr/FreeBSD-CVS/src/sys/netinet/ip_input.c,v
 > > retrieving revision 1.111
 > > diff -u -r1.111 ip_input.c
 > > --- ip_input.c	1999/01/12 12:25:00	1.111
 > > +++ ip_input.c	1999/03/10 21:37:33
 > > @@ -1436,6 +1436,8 @@
 > >  	 * we need to generate an ICMP message to the src.
 > >  	 */
 > >  	mcopy = m_copy(m, 0, imin((int)ip->ip_len, 64));
 > > +	if (mcopy && (mcopy->m_flags & M_EXT))
 > > +		mcopy = m_pullup(mcopy, mcopy->m_len);
 >  
 > >  	/*
 > >  	 * If forwarding packet using same interface that it came in on,
 > 
 > This does look like it does as you suggest, but I don't think it's the
 > right behavior -- we don't want to be hand-copying every packet as we
 > send it.  The Right Thing is substantially more complicated; what we
 > need to do is to change the interface to ip_output() (and by extension
 > to if_output()) such that the original mbuf chain is not freed on
 > error.  Then, ip_output would undo its manipulation of the packet (and
 > expect the caller to free it after doing whatever was necessary).
 > This would also speed up the general case of forwarding.
 > 
 > -GAWollman
 
 Anyway, ip_output() will change the byte order from host to net,
 so the problem will remain and icmp_error() will fail.  Right?
 
 -- 
 Ruslan Ermilov		Sysadmin and DBA of the
 ru@ucb.crimea.ua	United Commercial Bank
 +380.652.247.647	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: from ark.cris.net (ark.cris.net [212.110.128.68])
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) with ESMTP id UAA18015
 	for <ru@ucb.crimea.ua>; Thu, 11 Mar 1999 20:51:02 +0200 (EET)
 	(envelope-from fenner@research.att.com)
 Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.30.102])
 	by ark.cris.net (8.8.8/8.8.8) with ESMTP id UAA06535
 	for <ru@ucb.crimea.ua>; Thu, 11 Mar 1999 20:50:08 +0200 (EET)
 Received: from alliance.research.att.com (alliance.research.att.com [135.207.26.26])
 	by mail-blue.research.att.com (Postfix) with ESMTP
 	id 2AC694CE2A; Thu, 11 Mar 1999 13:46:52 -0500 (EST)
 Received: from windsor.research.att.com (windsor.research.att.com [135.207.26.46])
 	by alliance.research.att.com (8.8.7/8.8.7) with ESMTP id NAA17394;
 	Thu, 11 Mar 1999 13:46:51 -0500 (EST)
 From: Bill Fenner <fenner@research.att.com>
 Received: (from fenner@localhost)
 	by windsor.research.att.com (8.8.7/8.8.5) id NAA19297;
 	Thu, 11 Mar 1999 13:46:50 -0500 (EST)
 Message-Id: <199903111846.NAA19297@windsor.research.att.com>
 MIME-Version: 1.0
 Content-Type: text/plain; charset=US-ASCII
 To: wollman@khavrinen.lcs.mit.edu
 Subject: Re: ip_input.c bug!!!
 Cc: ru@ucb.crimea.ua, wollman@freebsd.org, fenner@freebsd.org,
         core@freebsd.org
 References:  <19990311122858.A77664@relay.ucb.crimea.ua> <199903111717.MAA01325@khavrinen.lcs.mit.edu>
 Date: Thu, 11 Mar 1999 13:46:50 -0500
 Versions: dmail (solaris) 2.2b/makemail 2.8t
 
 
 >This does look like it does as you suggest, but I don't think it's the
 >right behavior -- we don't want to be hand-copying every packet as we
 >send it.  The Right Thing is substantially more complicated; what we
 >need to do is to change the interface to ip_output() (and by extension
 >to if_output()) such that the original mbuf chain is not freed on
 >error.
 
 I agree.  A couple of years ago I went through the ICMP stuff looking for
 things to clean up and decided that although there were several
 inconsistencies, that the interface change was the only way to solve
 it (and it's more complex than Garrett mentions, since the interface
 may have modified the mbuf chain before discovering the error so
 your original pointer may not be valid, or the original data might
 have moved, or...)
 
   Bill
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: (from ru@localhost)
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) id LAA99823;
 	Fri, 12 Mar 1999 11:40:29 +0200 (EET)
 	(envelope-from ru)
 Date: Fri, 12 Mar 1999 11:40:29 +0200
 From: Ruslan Ermilov <ru@ucb.crimea.ua>
 To: Bill Fenner <fenner@research.att.com>
 Cc: wollman@khavrinen.lcs.mit.edu, core@freebsd.org
 Subject: Re: ip_input.c bug!!!
 Message-ID: <19990312114029.B91406@relay.ucb.crimea.ua>
 References: <19990311122858.A77664@relay.ucb.crimea.ua> <199903111717.MAA01325@khavrinen.lcs.mit.edu> <199903111846.NAA19297@windsor.research.att.com>
 Mime-Version: 1.0
 Content-Type: text/plain; charset=us-ascii
 X-Mailer: Mutt 0.95.3i
 In-Reply-To: <199903111846.NAA19297@windsor.research.att.com>; from Bill Fenner on Thu, Mar 11, 1999 at 01:46:50PM -0500
 X-Operating-System: FreeBSD 3.1-STABLE i386
 
 On Thu, Mar 11, 1999 at 01:46:50PM -0500, Bill Fenner wrote:
 > 
 > >This does look like it does as you suggest, but I don't think it's the
 > >right behavior -- we don't want to be hand-copying every packet as we
 > >send it.  The Right Thing is substantially more complicated; what we
 > >need to do is to change the interface to ip_output() (and by extension
 > >to if_output()) such that the original mbuf chain is not freed on
 > >error.
 > 
 > I agree.  A couple of years ago I went through the ICMP stuff looking for
 > things to clean up and decided that although there were several
 > inconsistencies, that the interface change was the only way to solve
 > it (and it's more complex than Garrett mentions, since the interface
 > may have modified the mbuf chain before discovering the error so
 > your original pointer may not be valid, or the original data might
 > have moved, or...)
 > 
 >   Bill
 
 While you are talking about long-term solution, let's just fix the
 bug meanwhile.
 
 The following patch just adds another XXX.
 
 
 Index: ip_input.c
 ===================================================================
 RCS file: /usr/FreeBSD-CVS/src/sys/netinet/ip_input.c,v
 retrieving revision 1.111
 diff -u -r1.111 ip_input.c
 --- ip_input.c	1999/01/12 12:25:00	1.111
 +++ ip_input.c	1999/03/12 09:00:17
 @@ -1471,6 +1471,16 @@
  
  	error = ip_output(m, (struct mbuf *)0, &ipforward_rt, 
  			  IP_FORWARDING, 0);
 +
 +	/*
 +	 * XXX ip_output() may have changed the byte order of fields in mcopy
 +	 */
 +	if (mcopy && (mcopy->m_flags & M_EXT)) {
 +		struct ip *ipcopy = mtod(mcopy, struct ip *);
 +		NTOHS(ipcopy->ip_len);
 +		NTOHS(ipcopy->ip_off);
 +        }
 +
  	if (error)
  		ipstat.ips_cantforward++;
  	else {
 
 
 
 
 Cheers,
 -- 
 Ruslan Ermilov		Sysadmin and DBA of the
 ru@ucb.crimea.ua	United Commercial Bank
 +380.652.247.647	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.30.103])
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) with ESMTP id VAA49802
 	for <ru@ucb.crimea.ua>; Fri, 12 Mar 1999 21:54:39 +0200 (EET)
 	(envelope-from fenner@research.att.com)
 Received: from alliance.research.att.com (alliance.research.att.com [135.207.26.26])
 	by mail-green.research.att.com (Postfix) with ESMTP
 	id 455701E032; Fri, 12 Mar 1999 14:54:31 -0500 (EST)
 Received: from windsor.research.att.com (windsor.research.att.com [135.207.26.46])
 	by alliance.research.att.com (8.8.7/8.8.7) with ESMTP id OAA16284;
 	Fri, 12 Mar 1999 14:54:23 -0500 (EST)
 From: Bill Fenner <fenner@research.att.com>
 Received: (from fenner@localhost)
 	by windsor.research.att.com (8.8.7/8.8.5) id OAA28324;
 	Fri, 12 Mar 1999 14:54:21 -0500 (EST)
 Message-Id: <199903121954.OAA28324@windsor.research.att.com>
 MIME-Version: 1.0
 Content-Type: text/plain; charset=US-ASCII
 To: ru@ucb.crimea.ua
 Subject: Re: ip_input.c bug!!!
 Cc: fenner@research.att.com, wollman@khavrinen.lcs.mit.edu, core@freebsd.org
 References:  <19990311122858.A77664@relay.ucb.crimea.ua> <199903111717.MAA01325@khavrinen.lcs.mit.edu> <199903111846.NAA19297@windsor.research.att.com> <19990312114029.B91406@relay.ucb.crimea.ua>
 Date: Fri, 12 Mar 1999 11:54:21 -0800
 Versions: dmail (solaris) 2.2b/makemail 2.8t
 
 
 Your XXX comment points out the problem with this patch; ip_output
 *sometimes* swaps the packet.  For things like EHOSTUNREACH, it
 doesn't, so this change makes it wrong for those errors.
 
   Bill
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193])
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) with ESMTP id WAA51522
 	for <ru@ucb.crimea.ua>; Fri, 12 Mar 1999 22:02:51 +0200 (EET)
 	(envelope-from wollman@khavrinen.lcs.mit.edu)
 Received: (from wollman@localhost)
 	by khavrinen.lcs.mit.edu (8.9.1/8.9.1) id PAA06727;
 	Fri, 12 Mar 1999 15:02:41 -0500 (EST)
 	(envelope-from wollman)
 Date: Fri, 12 Mar 1999 15:02:41 -0500 (EST)
 From: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
 Message-Id: <199903122002.PAA06727@khavrinen.lcs.mit.edu>
 To: Bill Fenner <fenner@research.att.com>
 Cc: ru@ucb.crimea.ua, wollman@khavrinen.lcs.mit.edu
 Subject: Re: ip_input.c bug!!!
 In-Reply-To: <199903121954.OAA28324@windsor.research.att.com>
 References: <19990311122858.A77664@relay.ucb.crimea.ua>
 	<199903111717.MAA01325@khavrinen.lcs.mit.edu>
 	<199903111846.NAA19297@windsor.research.att.com>
 	<19990312114029.B91406@relay.ucb.crimea.ua>
 	<199903121954.OAA28324@windsor.research.att.com>
 
 <<On Fri, 12 Mar 1999 11:54:21 -0800, Bill Fenner <fenner@research.att.com> said:
 
 > Your XXX comment points out the problem with this patch; ip_output
 > *sometimes* swaps the packet.  For things like EHOSTUNREACH, it
 > doesn't, so this change makes it wrong for those errors.
 
 And, what's worse, if_output() could conceivably return EHOSTUNREACH,
 so you can't use the error to distinguish these cases.
 
 -GAWollman
 
 --
 Garrett A. Wollman   | O Siem / We are all family / O Siem / We're all the same
 wollman@lcs.mit.edu  | O Siem / The fires of freedom 
 Opinions not those of| Dance in the burning flame
 MIT, LCS, CRS, or NSA|                     - Susan Aglukark and Chad Irschick
 
 --azLHFNyN32YCQGCU
 Content-Type: message/rfc822
 
 Received: (from ru@localhost)
 	by relay.ucb.crimea.ua (8.9.2/8.9.2/UCB) id WAA55540;
 	Fri, 12 Mar 1999 22:25:43 +0200 (EET)
 	(envelope-from ru)
 Date: Fri, 12 Mar 1999 22:25:42 +0200
 From: Ruslan Ermilov <ru@ucb.crimea.ua>
 To: Bill Fenner <fenner@research.att.com>
 Cc: wollman@khavrinen.lcs.mit.edu, core@freebsd.org
 Subject: Re: ip_input.c bug!!!
 Message-ID: <19990312222542.A53502@relay.ucb.crimea.ua>
 References: <19990311122858.A77664@relay.ucb.crimea.ua> <199903111717.MAA01325@khavrinen.lcs.mit.edu> <199903111846.NAA19297@windsor.research.att.com> <19990312114029.B91406@relay.ucb.crimea.ua> <199903121954.OAA28324@windsor.research.att.com>
 Mime-Version: 1.0
 Content-Type: text/plain; charset=us-ascii
 X-Mailer: Mutt 0.95.3i
 In-Reply-To: <199903121954.OAA28324@windsor.research.att.com>; from Bill Fenner on Fri, Mar 12, 1999 at 11:54:21AM -0800
 X-Operating-System: FreeBSD 3.1-STABLE i386
 
 On Fri, Mar 12, 1999 at 11:54:21AM -0800, Bill Fenner wrote:
 > 
 > Your XXX comment points out the problem with this patch; ip_output
 > *sometimes* swaps the packet.  For things like EHOSTUNREACH, it
 > doesn't, so this change makes it wrong for those errors.
 > 
 >   Bill
 
 How about the following?
 
 
 Index: ip_input.c
 ===================================================================
 RCS file: /usr/FreeBSD-CVS/src/sys/netinet/ip_input.c,v
 retrieving revision 1.111
 diff -u -r1.111 ip_input.c
 --- ip_input.c	1999/01/12 12:25:00	1.111
 +++ ip_input.c	1999/03/12 20:22:30
 @@ -1388,6 +1388,7 @@
  	register struct rtentry *rt;
  	int error, type = 0, code = 0;
  	struct mbuf *mcopy;
 +	u_short ip__off, ip__len;
  	n_long dest;
  	struct ifnet *destifp;
  
 @@ -1469,8 +1470,20 @@
  		}
  	}
  
 +	ip__off = ip->ip_off;
 +	ip__len = ip->ip_len;
  	error = ip_output(m, (struct mbuf *)0, &ipforward_rt, 
  			  IP_FORWARDING, 0);
 +
 +	/*
 +	 * XXX ip_output() may have changed the byte order of fields in mcopy
 +	 */
 +	if (mcopy && (mcopy->m_flags & M_EXT)) {
 +		struct ip *ipcopy = mtod(mcopy, struct ip *);
 +		ipcopy->ip_len = ip__len;
 +		ipcopy->ip_off = ip__off;
 +        }
 +
  	if (error)
  		ipstat.ips_cantforward++;
  	else {
 
 
 
 -- 
 Ruslan Ermilov		Sysadmin and DBA of the
 ru@ucb.crimea.ua	United Commercial Bank
 +380.652.247.647	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
 --azLHFNyN32YCQGCU--
 

From: Kannan Varadhan <kannanv@malgudi.research.bell-labs.com>
To: Ruslan Ermilov <ru@FreeBSD.org>
Cc: kannanv@research.bell-labs.com,
	Garrett Wollman <wollman@FreeBSD.org>,
	freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/16240: ICMP error generation fails to correctly insert IP ID on returned packet 
Date: Fri, 21 Jan 2000 12:12:53 -0500

 Excuse me for being stupid here, but what does ping from SCO hosts have
 to do with how freebsd behaves?  Here's a different synopsis of the bug:
 
    freebsd host, call it X, decides to send an ICMP error message.
    It has to copy back some portion of the original packet that caused
       the error.
    During the copy process, the ip_d field of the original packet is not
       in network byte order.
 
 Here's another look at the tcpdump trace:
 
 > ### ORIGINATING PACKET.  NOTE IP ID is 0194
 > 16:31:54.085012 0:50:4:b1:f0:90 0:60:1d:9:0:5a 0800 154: 135.104.73.11.2049 > 135.104.73.82.2094666972: reply ok 112 getattr [|nfs]
 >                          4500 008c 0194 0000 4011 d79f 8768 490b
                                      ^^^^ ORIGINAL ip_id
 >                          8768 4952 0801 03e5 0078 152e 7cda 14dc
 >                          0000 0001 0000 0000 0000 0000 0000 0000
 >                          0000 0000 0000
 > 
 > ### ICMP ERROR RETURNED.  NOTE IP ID on returned packet is 9401,
 > ### indicating some byte-ordering problems.
 > 16:31:54.085486 0:60:1d:9:0:5a 0:50:4:b1:f0:90 0800 70: 135.104.73.82 > 135.104.73.11: icmp: 135.104.73.82 udp port 997 unreachable
 >                          4500 0038 012e 0000 fd01 1b69 8768 4952
 >                          8768 490b 0303 5e31 0000 0000 4500 008c
 >                          9401 0000 3e11 d99f 8768 490b 8768 4952
                            ^^^^ Copied ip_id.
 >                          0801 03e5 0078
 
 Looking over the icmp_error code in /sys/netinet/ip_icmp.c, towards the
 end where it copies the original packet into the icmp packet, we have:
 
     179         icp->icmp_code = code;
     180         bcopy((caddr_t)oip, (caddr_t)&icp->icmp_ip, icmplen);
     181         nip = &icp->icmp_ip;
     182         nip->ip_len = htons((u_short)(nip->ip_len + oiplen));
     183         
 
 Clearly, the code goes to the trouble of making sure the ip_len is in
 network byte order.  Why does it not do the same for nip->ip_id?  Is
 that not sufficient?
 
 
 Kannan
 

From: Ruslan Ermilov <ru@FreeBSD.org>
To: Kannan Varadhan <kannanv@malgudi.research.bell-labs.com>
Cc: kannanv@research.bell-labs.com,
	Garrett Wollman <wollman@FreeBSD.org>,
	freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/16240: ICMP error generation fails to correctly insert IP ID on returned packet
Date: Fri, 21 Jan 2000 20:15:19 +0200

 On Fri, Jan 21, 2000 at 12:12:53PM -0500, Kannan Varadhan wrote:
 > Excuse me for being stupid here, but what does ping from SCO hosts have
 > to do with how freebsd behaves?  Here's a different synopsis of the bug:
 > 
 Ping from SCO differ in that SCO sets DF bit in ICMP packets, which
 triggers the same bug as you observe (the bytes in ICMP response
 packets appear in the host order).
 
 -- 
 Ruslan Ermilov		Sysadmin and DBA of the
 ru@ucb.crimea.ua	United Commercial Bank,
 ru@FreeBSD.org		FreeBSD committer,
 +380.652.247.647	Simferopol, Ukraine
 
 http://www.FreeBSD.org	The Power To Serve
 http://www.oracle.com	Enabling The Information Age
 
State-Changed-From-To: open->analyzed 
State-Changed-By: ru 
State-Changed-When: Wed Aug 30 01:44:12 PDT 2000 
State-Changed-Why:  
The patch is being developed. 


Responsible-Changed-From-To: freebsd-bugs->ru 
Responsible-Changed-By: ru 
Responsible-Changed-When: Wed Aug 30 01:44:12 PDT 2000 
Responsible-Changed-Why:  
I am working on the patch. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=16240 
State-Changed-From-To: analyzed->feedback 
State-Changed-By: ru 
State-Changed-When: Fri Sep 1 05:47:54 PDT 2000 
State-Changed-Why:  
Fixed in 5.0-CURRENT in the following files/revisions: 
src/sys/netinet/ip_divert.c, rev 1.46 
src/sys/netinet/ip_icmp.c, rev 1.44 
src/sys/netinet/ip_input.c, rev 1.139 
src/sys/netinet/ip_mroute.c, rev 1.58 
src/sys/netinet/ip_output.c, rev 1.110 
src/sys/netinet/raw_ip.c, rev 1.68 
src/sys/netinet/udp_usrreq.c, rev 1.73 

http://www.freebsd.org/cgi/query-pr.cgi?pr=16240 
State-Changed-From-To: feedback->closed 
State-Changed-By: ru 
State-Changed-When: Tue Sep 26 10:27:23 PDT 2000 
State-Changed-Why:  
Fixed in 4.1-STABLE. 
Won't be merged onto RELENG_3 branch, sorry. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=16240 
>Unformatted:
