From fujiwara@wide.ad.jp  Sun Sep 19 17:39:03 2004
Return-Path: <fujiwara@wide.ad.jp>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 7F56D16A4CF
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 19 Sep 2004 17:39:03 +0000 (GMT)
Received: from f.pyon.org (i220-220-111-153.s05.a013.ap.plala.or.jp [220.220.111.153])
	by mx1.FreeBSD.org (Postfix) with SMTP id AC5CE43D5C
	for <FreeBSD-gnats-submit@freebsd.org>; Sun, 19 Sep 2004 17:39:02 +0000 (GMT)
	(envelope-from fujiwara@wide.ad.jp)
Received: (qmail 1208 invoked from network); 19 Sep 2004 17:39:01 -0000
Received: from localhost6 (HELO localhost) (0000:0000:0000:0000:0000:0000:0000:0001)
  by localhost6 with SMTP; 19 Sep 2004 17:39:01 -0000
Message-Id: <20040920.023859.74753252.fujiwara@pyon.org>
Date: Mon, 20 Sep 2004 02:38:59 +0900 (JST)
From: Kazunori_Fujiwara <fujiwara@wide.ad.jp>
Reply-To: Kazunori_Fujiwara <fujiwara@wide.ad.jp>
To: FreeBSD-gnats-submit@freebsd.org
Subject: ipfw forward does not work
X-Send-Pr-Version: 3.113
X-GNATS-Notify: ru

>Number:         71910
>Category:       kern
>Synopsis:       [patch] ipfw forward does not work [5.3-BETA4]
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    andre
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Sep 19 17:40:23 GMT 2004
>Closed-Date:    Sat Sep 10 07:48:54 GMT 2005
>Last-Modified:  Sat Sep 10 07:48:54 GMT 2005
>Originator:     Kazunori_Fujiwara
>Release:        FreeBSD 5.3-BETA4 i386 and 6-CURRENT
>Organization:
JPRS/WIDE Project
>Environment:
System: FreeBSD f.pyon.org 5.3-BETA4 FreeBSD 5.3-BETA4 #5: Sat Sep 18 11:44:33 JST 2004 fujiwara@f.pyon.org:/usr/obj/usr/src/sys/SC5 i386
System: FreeBSD tree.private 6.0-CURRENT FreeBSD 6.0-CURRENT #8: Mon Sep 20 00:38:18 JST 2004 fujiwara@tree.private:/usr/src/sys/i386/compile/TREE i386

>Description:

  'ipfw fwd' don't forward packet for specified destination
when packet ip_src is the same machine's.

  I checked this on 5.3-BETA4 and 6-current

>How-To-Repeat:

  enable kernel option IPFIREWALL and IPFIREWALL_FORWARD

  ipfw add fwd remote ip from myaddr to any

>Fix:

Index: ip_output.c
===================================================================
RCS file: /FreeBSD-CVS/src/sys/netinet/ip_output.c,v
retrieving revision 1.225.2.3
diff -u -b -r1.225.2.3 ip_output.c
--- ip_output.c	15 Sep 2004 15:07:09 -0000	1.225.2.3
+++ ip_output.c	19 Sep 2004 15:51:42 -0000
@@ -713,7 +713,7 @@
 	/* Or forward to some other address? */
 	fwd_tag = m_tag_find(m, PACKET_TAG_IPFORWARD, NULL);
 	if (fwd_tag) {
-		if (!in_localip(ip->ip_src) && !in_localaddr(ip->ip_dst)) {
+		if (!in_localaddr(ip->ip_dst)) {
 			dst = (struct sockaddr_in *)&ro->ro_dst;
 			bcopy((fwd_tag+1), dst, sizeof(struct sockaddr_in));
 			m->m_flags |= M_SKIP_FIREWALL;
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->andre 
Responsible-Changed-By: ru 
Responsible-Changed-When: Mon Sep 20 07:14:47 GMT 2004 
Responsible-Changed-Why:  
Redirect to Andre. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71910 

From: John Hay <jhay@icomtek.csir.co.za>
To: Kazunori_Fujiwara <fujiwara@wide.ad.jp>
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/71910: ipfw forward does not work
Date: Mon, 27 Sep 2004 21:38:54 +0200

 > >Description:
 > 
 >   'ipfw fwd' don't forward packet for specified destination
 > when packet ip_src is the same machine's.
 > 
 >   I checked this on 5.3-BETA4 and 6-current
 > 
 
 I also ran into this problem when upgrading. I have a machine that is
 a NAT box and also have a rule to fwd port 80 connections to our web
 proxy machine for transparent proxying. Without this patch the fwd
 rule didn't work, but it is working fine with the patch applied.
 
 John
 -- 
 John Hay -- John.Hay@icomtek.csir.co.za / jhay@FreeBSD.org

From: Saulius Menkevicius <bob@nulis.lt>
To: freebsd-gnats-submit@FreeBSD.org, fujiwara@wide.ad.jp
Cc:  
Subject: Re: kern/71910: [patch] ipfw forward does not work [5.3-BETA4]
Date: Sat, 02 Oct 2004 20:46:49 +0300

 This patch fixes "ipfw fwd" for me too.
 I implement some sort of outbound load-balancing on two dsl lines by 
 forwarding some of the packets to another gateway with my ipfw setup.
 
 FreeBSD nulis.lt 5.3-BETA6 FreeBSD 5.3-BETA6 #2: Sat Oct  2 03:48:29 
 EEST 2004     bob@nulis.lt:/usr/src/sys/i386/compile/MIDWAY  i386

From: Yoshiaki Kasahara <kasahara@nc.kyushu-u.ac.jp>
To: freebsd-gnats-submit@FreeBSD.org, fujiwara@wide.ad.jp
Cc:  
Subject: Re: kern/71910: [patch] ipfw forward does not work [5.3-BETA4]
Date: Tue, 07 Dec 2004 19:51:22 +0900 (JST)

 The problem still exists on FreeBSD-5.3-STABLE system, and the patch
 fixes my problem too.
 
 I have a multihomed server which connects to different networks, and I
 want to select the interface for outgoing packets based on incoming
 connection. Without this patch 'ipfw fwd' won't forward the outgoing
 packet correctly.
 
 FreeBSD elvenbow.nc.kyushu-u.ac.jp 5.3-STABLE FreeBSD 5.3-STABLE #1: Mon Dec  6 20:11:17 JST 2004     root@elvenbow.nc.kyushu-u.ac.jp:/usr/obj/usr/src/sys/ELVENBOW  i386
 
 -- 
 Yoshiaki Kasahara
 kasahara@nc.kyushu-u.ac.jp
 

From: "Nick George" <nick.george@gmail.com>
To: <freebsd-gnats-submit@FreeBSD.org>, <fujiwara@wide.ad.jp>
Cc:  
Subject: Re: kern/71910: [patch] ipfw forward does not work [5.3-BETA4]
Date: Tue, 7 Dec 2004 11:30:37 -0500

 This is a multi-part message in MIME format.
 
 ------=_NextPart_000_0012_01C4DC50.2C6D1FE0
 Content-Type: text/plain;
 	charset="iso-8859-1"
 Content-Transfer-Encoding: quoted-printable
 
 I have run into a closely related problem, however the suggested patch =
 does not fix it.
 
 ipfw fwd will not forward packets to the specified destination when the =
 packet ip_dst is the same as the ip address of the host running the =
 firewall.=20
 
 The packet counters in ipfw are incremented, but the packet never makes =
 it out the interface. The packets are instead passed to the local host.=20
 
 To reproduce: =20
 ipfw add 1 fwd <remote ip> ip from any to <local ip>
 
 
 ------=_NextPart_000_0012_01C4DC50.2C6D1FE0
 Content-Type: text/html;
 	charset="iso-8859-1"
 Content-Transfer-Encoding: quoted-printable
 
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
 <HTML><HEAD>
 <META http-equiv=3DContent-Type content=3D"text/html; =
 charset=3Diso-8859-1">
 <META content=3D"MSHTML 6.00.2900.2523" name=3DGENERATOR>
 <STYLE></STYLE>
 </HEAD>
 <BODY bgColor=3D#ffffff>
 <DIV><FONT face=3DArial size=3D2>I have run into a closely related =
 problem, however=20
 the suggested patch does not fix it.</FONT></DIV>
 <DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
 <DIV><FONT face=3DArial size=3D2>ipfw fwd will not forward packets to =
 the specified=20
 destination when the packet <STRONG>ip_dst</STRONG> is the same as the =
 ip=20
 address of the host running the firewall. </FONT></DIV>
 <DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
 <DIV><FONT face=3DArial size=3D2>The packet counters in ipfw are =
 incremented, but=20
 the packet never makes it out the interface. The packets are instead =
 passed to=20
 the local host. </FONT></DIV>
 <DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
 <DIV><FONT face=3DArial size=3D2>To reproduce: &nbsp;</FONT></DIV>
 <DIV><FONT face=3DArial size=3D2>ipfw add 1 fwd &lt;remote ip&gt; ip =
 from any to=20
 &lt;local ip&gt;</FONT></DIV>
 <DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
 <DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV></BODY></HTML>
 
 ------=_NextPart_000_0012_01C4DC50.2C6D1FE0--
 

From: Dmitry Dolzenko <dol@mig.phys.msu.ru>
To: freebsd-gnats-submit@FreeBSD.org, fujiwara@wide.ad.jp
Cc:  
Subject: Re: kern/71910: [patch] ipfw forward does not work [5.3-BETA4]
Date: Thu, 27 Jan 2005 21:37:42 +0300

 Hello,
 
  The problem still exists on  FreeBSD 5.3-RELEASE-p5 system, and the patch
  fixes my problem too.
 
  I have a dual link Internet connection server. Squid proxy configured
  to forward requests for some clients with different source IP
  addresses. This IP address is local.
  Than this request forwarded by "ipfw fwd" to hi speed uplink.
  
  Without patch system don't forward packets correctly.
  
 
 -- 
 WBR Dmitry
 

From: Peter Much <pmc@citylink.dinoex.sub.org>
To: freebsd-gnats-submit@FreeBSD.org, fujiwara@wide.ad.jp
Cc:  
Subject: Re: kern/71910: [patch] ipfw forward does not work [5.3-BETA4]
Date: Sun, 20 Feb 2005 17:37:18 +0100

 Hi all,
 
   thanx for the patch, it fixed the forward so far that my most
 important activity did get operational. :-)
 
   But the forwarding is still not working correct. There are at
 least two remaining bugs (that is, the two I stumbled into by 
 accident). I am running 5.3-p5, and I verified both do not appear
 with RELEASE-4.7. 
 
 First:
   Nick George noticed it already. More completely said:
 If the actual destination address of an outgoing packet is some-
 where on a locally attached network, and it shall get routed out
 thru some other interface by means of "ipfw fwd", then this fwd 
 will get silently ignored and treated as an "accept". That is,
 the packet will leave ipfw at this rule, and will NOT be routed
 according to the address given in "ipfw fwd", but instead to the
 interface that was found in the routing table.
 
 This is easy to reproduce even on a productive system: create a 
 tunnel or netgraph interface with a pair of RFC1918 addresses, 
 forward some unused service to that external adress, and watch 
 with "netstat -i" if the counters increase when trying to contact
 this service on various addresses within local netmasks.
 
 Second:
   When an "ipfw forward" directs a packet onto a netgraph iface,
 where this netgraph iface works as a loopback and the packet should
 be received back into the local system, then this packet will
 be moved into netgraph, and will be received back from netgraph,
 and then it vanishes! It will not be seen by ipfw again as 
 received, neither will it be seen again by anybody else.
 That is, the sent counter of the respective netgraph iface 
 increases, the received counter of the respective netgraph iface
 increases, and thats it.
 The loopback can be implemented by two netgraph ifaces, or by
 one netgraph iface connected to a netgraph echo. If the same 
 loopback is implemented with a tun device and 
 "cat /dev/tun2 > /dev/tun2", then the problem does NOT appear,
 i.e. then it works as expected.
 If packets are routed into such a netgraph loopback not per 
 "ipfw fwd", but per normal routing table, then the problem also
 does NOT appear, i.e. then the obvious endless loop will happen.
 
 This bug is also easy to reproduce, in the following way:
   1. Create the netgraph objects:
     $ ngctl
     > mkpeer . iface tmp ipx
     > mkpeer .:tmp iface inet inet
     > rmhook . tmp
   2. put a suitable adress onto the interface:
     $ ifconfig ng0 192.168.37.1 192.168.37.2
     $ ifconfig ng1 up
     (the second interface is only receiving, it doesnt need an adress)
   3. put some rules into ipfw (here we try it with http):
     $ ipfw add 10 count log tcp from any to any 80
     $ ipfw add 11 skipto 13 log ip from any to any out recv ng1
     $ ipfw add 12 fwd 192.168.37.2 log tcp from any to any 80 out
   4. watch the packets match rule 12 and then not reappear:
     $ telnet www.google.de 80
 
 Why do I need the functionality: I use a stateful firewall with
 natd, so I must keepstate the incoming packets. But an adress to
 be nat'ed is actually a jail on the local system, so there are
 no incoming packets. With this construct I make them incoming.
 -- 
 PMc

From: Fabian Wenk <fabian@wenks.ch>
To: bug-followup@FreeBSD.org, fujiwara@wide.ad.jp
Cc:  
Subject: Re: kern/71910: [patch] ipfw forward does not work [5.3-BETA4]
Date: Sun, 22 May 2005 22:33:32 +0200

 Hello
 
 I just set up a dual homed box and tried it out with FreeBSD 
 5.4-RELEASE-p1 with the following options in the kernel:
 
 options   IPFIREWALL
 options   IPFIREWALL_VERBOSE
 options   IPFIREWALL_VERBOSE_LIMIT=512
 options   IPFIREWALL_DEFAULT_TO_ACCEPT
 options   IPFIREWALL_FORWARD
 options   IPFIREWALL_FORWARD_EXTENDED
 options   IPDIVERT
 
 In the first place I left out the IPFIREWALL_FORWARD_EXTENDED 
 options and it was not working. But after a look at the source in 
 the place where the patch from this report should have gone, I 
 put it in and everything was working as expected, so the patching 
 is not needed any more.
 
 But be careful with the IPFIREWALL_FORWARD_EXTENDED option, see 
 /usr/src/sys/conf/NOTES. In my setup the 
 IPFIREWALL_DEFAULT_TO_ACCEPT option probably does the trick, I 
 use it anyway.
 
 
 bye
 Fabian
State-Changed-From-To: open->closed 
State-Changed-By: andre 
State-Changed-When: Sat Sep 10 07:48:22 GMT 2005 
State-Changed-Why:  
Solved by enabling IPFIREWALL_FORWARD_EXTENDED kernel option. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=71910 
>Unformatted:
