From nobody@FreeBSD.org  Mon Oct  8 02:14:19 2001
Return-Path: <nobody@FreeBSD.org>
Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id B18C137B405
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  8 Oct 2001 02:14:18 -0700 (PDT)
Received: (from nobody@localhost)
	by freefall.freebsd.org (8.11.4/8.11.4) id f989EIh38572;
	Mon, 8 Oct 2001 02:14:18 -0700 (PDT)
	(envelope-from nobody)
Message-Id: <200110080914.f989EIh38572@freefall.freebsd.org>
Date: Mon, 8 Oct 2001 02:14:18 -0700 (PDT)
From: Tim Burgess <tburgess@whitley.unimelb.edu.au>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ipfw tee functionality causes malfunction and security hole
X-Send-Pr-Version: www-1.0

>Number:         31130
>Category:       kern
>Synopsis:       ipfw tee functionality causes malfunction and security hole
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    cjc
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Oct 08 02:20:02 PDT 2001
>Closed-Date:    Sat Feb 9 02:48:33 PST 2002
>Last-Modified:  Sat Feb 09 02:49:08 PST 2002
>Originator:     Tim Burgess
>Release:        4.2-RELEASE
>Organization:
Whitley College
>Environment:
FreeBSD death.whitley.unimelb.edu.au 4.2-RELEASE FreeBSD 4.2-RELEASE #4: Tue Sep
  4 21:42:46 EST 2001     tburgess@death.whitley.unimelb.edu.au:/usr/src/sys/com
pile/DEATH  i386
>Description:
It looks to me like using the ipfw 'tee' function on incoming packets actually accepts the packets as destined for the localhost.  Hence a rule such as:

600 tee 8665 ip from any to any in

Means that anyone browsing the web on the subnet behind the gateway sees the gateway machine's webserver no matter which url they enter.  www.hotmail.com/wi actually goes to www.whitley.unimelb.edu.au/wi !

Outgoing packets are fine.
>How-To-Repeat:
See above
>Fix:
Here is my experimental fix (untested, but it gives you an idea of what I think the problem is).  I will test it when I have a chance to bring down the (relatively important to us) machine for a little while.

(from netinet/ip_input.c)

/* If 'tee', continue with original packet */
if (clone == NULL)
  return;
m = clone;
ip = mtod(m, struct ip *);

Then normally it just keeps going in the 'ours' section of the code, which might explain the observed phenomena.  I think something along the lines of:

                /* TJB added this as an experimental bug fix */
                /* make sure we don't divert again - just accept the packet*/
                divert_info = 0;
                goto pass;

Note that ideally the packet SHOULD continue processing through the firewall but looking at the way the code is put together I can understand why this is nontrivial.  Oh well :(  Maybe it's possible to export the divert_out function from ip_divert.c and call it to reinject the duplicated packet immediately?
>Release-Note:
>Audit-Trail:

From: "Crist J. Clark" <cristjc@earthlink.net>
To: Tim Burgess <tburgess@whitley.unimelb.edu.au>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/31130: ipfw tee functionality causes malfunction and security hole
Date: Mon, 8 Oct 2001 03:20:15 -0700

 On Mon, Oct 08, 2001 at 02:14:18AM -0700, Tim Burgess wrote:
 
 [snip]
 
 > >Description:
 > It looks to me like using the ipfw 'tee' function on incoming packets actually accepts the packets as destined for the localhost.  Hence a rule such as:
 > 
 > 600 tee 8665 ip from any to any in
 > 
 > Means that anyone browsing the web on the subnet behind the gateway sees the gateway machine's webserver no matter which url they enter.  www.hotmail.com/wi actually goes to www.whitley.unimelb.edu.au/wi !
 
 I am not sure what you are saying here. The fact that the original
 packet is accepted is clearly documented in ipfw(8). Not ideal
 behavior, but documented behavior. As for this issue where you believe
 that you have redirected packets, what is listening on 8665/divert?
 Can we see a tcpdump(8) of this behavior?
 -- 
 Crist J. Clark                           cjclark@alum.mit.edu
                                          cjclark@jhu.edu
                                          cjc@freebsd.org

From: tburgess@whitley.unimelb.edu.au
To: cristjc@earthlink.net, tburgess@whitley.unimelb.edu.au
Cc: freebsd-gnats-submit@FreeBSD.ORG,
	tburgess-sent@whitley.unimelb.edu.au
Subject: Re: kern/31130: ipfw tee functionality causes malfunction and security hole
Date: Tue, 9 Oct 2001 12:15:52 +1000 (EST)

 OK, sorry for the cloudy explanation.
 
 Yeah ipfw says the packet is accepted.  But what i mean is:
 
 We have a machine with interfaces 203.5.71.6, 10.0.0.1 and others.  
 It forwards packets between the inside network and outside network, 
 through natd.  It also runs a webserver, squid proxy etc.
 
 if 10.0.1.53 (me) tries to access a webpage, say 64.1.2.3:80.  The 
 packet actually gets accepted by the apache server on the gateway 
 machine!!
 
 So the packet is 'accepted' but not in the normal sense of the 
 firewall 'accepting' the packet.  It's original destination is just 
 completely ignored.
 
 The daemon listening on 8665 is a traffic accounting program I have 
 written.  It does not inject any packets back into the socket.  In 
 fact, the behaviour is reproducible even when the userspace program 
 is not running (ie the packets from the tee are getting blackholed).
 
 Hope this helps to clarify things...
 
 I'll see if I can get a tcpdump of the behaviour in a few days.  
 Since it's a relatively important machine, I'll have to set up the 
 firewall rules to only create the problem for me and then do it.
 
 Thanks for your help,
 
 Tim
 
 
 ---- Original Message ----
 From:		Crist J. Clark
 Date:		Mon 10/8/01 20:24
 To:		Tim Burgess
 Cc:		freebsd-gnats-submit@FreeBSD.ORG
 Subject:	Re: kern/31130: ipfw tee functionality causes 
 malfunction and security hole
 
 On Mon, Oct 08, 2001 at 02:14:18AM -0700, Tim Burgess wrote:
 
 [snip]
 
 > >Description:
 > It looks to me like using the ipfw 'tee' function on incoming 
 packets actually accepts the packets as destined for the localhost.  
 Hence a rule such as:
 > 
 > 600 tee 8665 ip from any to any in
 > 
 > Means that anyone browsing the web on the subnet behind the gateway 
 sees the gateway machine's webserver no matter which url they enter.  
 www.hotmail.com/wi actually goes to www.whitley.unimelb.edu.au/wi !
 
 I am not sure what you are saying here. The fact that the original
 packet is accepted is clearly documented in ipfw(8). Not ideal
 behavior, but documented behavior. As for this issue where you believe
 that you have redirected packets, what is listening on 8665/divert?
 Can we see a tcpdump(8) of this behavior?
 -- 
 Crist J. Clark                           cjclark@alum.mit.edu
                                          cjclark@jhu.edu
                                          cjc@freebsd.org
 

From: tburgess@whitley.unimelb.edu.au
To: cristjc@earthlink.net, tburgess-sent@whitley.unimelb.edu.au
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/31130: ipfw tee functionality causes malfunction and security hole
Date: Tue, 9 Oct 2001 12:21:49 +1000 (EST)

 If it helps, this appears to be a more complicated description of the 
 same problem, a long time ago.
 
 http://docs.freebsd.org/cgi/getmsg.cgi?
 fetch=86560+0+archive/2000/freebsd-hackers/20000409.freebsd-hackers
 
 Kind regards,
 
 Tim
 
 ---- Original Message ----
 From:		Crist J. Clark
 Date:		Mon 10/8/01 20:24
 To:		Tim Burgess
 Cc:		freebsd-gnats-submit@FreeBSD.ORG
 Subject:	Re: kern/31130: ipfw tee functionality causes 
 malfunction and security hole
 
 On Mon, Oct 08, 2001 at 02:14:18AM -0700, Tim Burgess wrote:
 
 [snip]
 
 > >Description:
 > It looks to me like using the ipfw 'tee' function on incoming 
 packets actually accepts the packets as destined for the localhost.  
 Hence a rule such as:
 > 
 > 600 tee 8665 ip from any to any in
 > 
 > Means that anyone browsing the web on the subnet behind the gateway 
 sees the gateway machine's webserver no matter which url they enter.  
 www.hotmail.com/wi actually goes to www.whitley.unimelb.edu.au/wi !
 
 I am not sure what you are saying here. The fact that the original
 packet is accepted is clearly documented in ipfw(8). Not ideal
 behavior, but documented behavior. As for this issue where you believe
 that you have redirected packets, what is listening on 8665/divert?
 Can we see a tcpdump(8) of this behavior?
 -- 
 Crist J. Clark                           cjclark@alum.mit.edu
                                          cjclark@jhu.edu
                                          cjc@freebsd.org
 

From: "Crist J. Clark" <cristjc@earthlink.net>
To: tburgess@whitley.unimelb.edu.au
Cc: freebsd-gnats-submit@FreeBSD.ORG,
	tburgess-sent@whitley.unimelb.edu.au
Subject: Re: kern/31130: ipfw tee functionality causes malfunction and security hole
Date: Tue, 9 Oct 2001 02:14:17 -0700

 Yep. I can easily replicate this. If I ping a box with,
 
   01000 tee 2222 icmp from any to any
 
 I see,
 
   01:22:38.769793 0:c0:f0:5a:6c:a 0:90:27:13:25:40 0800 98: 192.168.64.60 > 172.16.0.1: icmp: echo request
   01:22:38.770281 0:90:27:13:25:40 0:c0:f0:5a:6c:a 0800 98: 192.168.64.30 > 192.168.64.60: icmp: echo reply
   01:22:39.776983 0:c0:f0:5a:6c:a 0:90:27:13:25:40 0800 98: 192.168.64.60 > 172.16.0.1: icmp: echo request
   01:22:39.777441 0:90:27:13:25:40 0:c0:f0:5a:6c:a 0800 98: 192.168.64.30 > 192.168.64.60: icmp: echo reply
   .
   .
   .
 
 On the wire and the packets never get routed to the "real" 172.16.0.1.
 Trying to figure out if,
 
   a) This is the expected behavior, but is poorly documented, or
   b) Something is broken.
 
 I'm thinking (b), but still wading through src/sys/netinet to verify.
 -- 
 Crist J. Clark                           cjclark@alum.mit.edu
                                          cjclark@jhu.edu
                                          cjc@freebsd.org

From: "Crist J. Clark" <cristjc@earthlink.net>
To: Tim Burgess <tburgess@whitley.unimelb.edu.au>
Cc: freebsd-gnats-submit@FreeBSD.ORG, freebsd-net@FreeBSD.ORG
Subject: Re: kern/31130: ipfw tee functionality causes malfunction and security hole
Date: Wed, 10 Oct 2001 02:35:47 -0700

 On Tue, Oct 09, 2001 at 02:20:02AM -0700, Crist J. Clark wrote:
 
 [snip]
 
 >  On the wire and the packets never get routed to the "real" 172.16.0.1.
 >  Trying to figure out if,
 >  
 >    a) This is the expected behavior, but is poorly documented, or
 >    b) Something is broken.
 >  
 >  I'm thinking (b), but still wading through src/sys/netinet to verify.
 
 Well, I see why this happens, but still not sure if it is supposed to
 happen. If we look at src/sys/netinet/ip_input.c, we see that all
 diverted or teed packets are accepted by the host as destined for
 itself,
 
 #ifdef IPDIVERT
 		if (i != 0 && (i & IP_FW_PORT_DYNT_FLAG) == 0) {
 			/* Divert or tee packet */
 			divert_info = i;
 			goto ours;
 		}
 #endif
 
 The packets are clearly going to be processed by the gateway. You seem
 to have already found this in the code snipped in your original PR,
 but I didn't notice your change. Please post changes to code as
 unified diffs. I now understand the 'fix' you were talking about. Have
 you actually built a kernel with your modifications? Does it seem to
 work?
 
 But packets _leaving_ the system seem to be processed as one would
 expect. That is, a copy is divert(4)ed and then the packet heads out
 onto to the wire.
 
 This apparent inconsistency is a bug since it is either unintended
 behavior or at least undocumented behavior. But the inconsistency
 gives you a temporary workaround. Instead of,
 
   600 tee 8665 ip from any to any in
 
 Does,
 
   600 tee 8665 ip from any to any out
 
 Work as you would expect?
 -- 
 Crist J. Clark                           cjclark@alum.mit.edu
                                          cjclark@jhu.edu
                                          cjc@freebsd.org

From: "Crist J. Clark" <cristjc@earthlink.net>
To: tburgess@whitley.unimelb.edu.au
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: kern/31130: ipfw tee functionality causes malfunction and security hole
Date: Sat, 13 Oct 2001 00:22:40 -0700

 This seems to fix 'tee' in ipfw(8) for me. Wanna give it a shot? BTW,
 I recently wrote a simple tool, Divert Packet Capture Daemon, dpcd,
 that catches packets on a divert(4) socket, passes them to a "module"
 that does arbitrary things, and then can (optionally) write the packet
 back. If you are still having 'tee' problems, let me know.
 
 Patch for -CURRENT. Should apply on -STABLE too.
 
 Index: src/sys/netinet/ip_input.c
 ===================================================================
 RCS file: /export/ncvs/src/sys/netinet/ip_input.c,v
 retrieving revision 1.182
 diff -u -r1.182 ip_input.c
 --- src/sys/netinet/ip_input.c	2001/10/05 05:45:27	1.182
 +++ src/sys/netinet/ip_input.c	2001/10/13 06:55:37
 @@ -794,6 +794,10 @@
  			return;
  		m = clone;
  		ip = mtod(m, struct ip *);
 +		/* XXX 'tee' was broken. Are these the right fixes? */
 +		ip->ip_len += hlen;
 +		divert_info = 0;
 +		goto pass;
  	}
  #endif
  
 
 -- 
 Crist J. Clark                     |     cjclark@alum.mit.edu
                                    |     cjclark@jhu.edu
 http://people.freebsd.org/~cjc/    |     cjc@freebsd.org
State-Changed-From-To: open->analyzed 
State-Changed-By: cjc 
State-Changed-When: Thu Nov 29 02:12:20 PST 2001 
State-Changed-Why:  
This is a bug. Testing patches. 


Responsible-Changed-From-To: freebsd-bugs->cjc 
Responsible-Changed-By: cjc 
Responsible-Changed-When: Thu Nov 29 02:12:20 PST 2001 
Responsible-Changed-Why:  
I'll commit a fix. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=31130 
State-Changed-From-To: analyzed->closed 
State-Changed-By: cjc 
State-Changed-When: Sat Feb 9 02:48:33 PST 2002 
State-Changed-Why:  
MFC'ed. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=31130 
>Unformatted:
