From nobody@FreeBSD.org  Wed Jun  9 12:23:32 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 0A6011065676
	for <freebsd-gnats-submit@FreeBSD.org>; Wed,  9 Jun 2010 12:23:32 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id EE5428FC1E
	for <freebsd-gnats-submit@FreeBSD.org>; Wed,  9 Jun 2010 12:23:31 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o59CNV03062419
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 9 Jun 2010 12:23:31 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id o59CNUql062418;
	Wed, 9 Jun 2010 12:23:30 GMT
	(envelope-from nobody)
Message-Id: <201006091223.o59CNUql062418@www.freebsd.org>
Date: Wed, 9 Jun 2010 12:23:30 GMT
From: Dmitriy Demidov <dima_bsd@inbox.lv>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ipfw dynamic rules and fwd
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         147720
>Category:       kern
>Synopsis:       [ipfw] ipfw dynamic rules and fwd
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-ipfw
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jun 09 12:30:04 UTC 2010
>Closed-Date:    Wed Jul 06 06:59:08 UTC 2011
>Last-Modified:  Tue Jul 12 16:20:08 UTC 2011
>Originator:     Dmitriy Demidov
>Release:        FreeBSD 7-STABLE
>Organization:
>Environment:
FreeBSD evo.local.home 7.3-PRERELEASE FreeBSD 7.3-PRERELEASE #0: Sat Feb 20 22:57:09 EET 2010 root@evo.local.home:/usr/obj/usr/src/sys/STABLE i386
>Description:
There is one bug with ipfw keep-state rules and fwd action.

You are unable to make source based routing of incoming connections using this ruleset:

=======
# $ext_if1 - ISP1
# $gw1 - ISP1 GW
# $ext_if2 - ISP2
# $gw2 - ISP2 GW
# $int_if - internal net

ipfw add 100 skipto 300 tag 1 in recv $ext_if1 keep-state
ipfw add 200 skipto 300 tag 2 in recv $ext_if2 keep-state
ipfw add 300 allow { recv $ext_if1 or recv $ext_if2 }
ipfw add 400 allow in recv $int_if
ipfw add 500 fwd $gw1 tagged 1
ipfw add 600 fwd $gw2 tagged 2
=======


To make it working you should patch /sys/netinet/ipfw/ip_fw2.c with this:

====
(find this line)
if (!q || dyn_dir == MATCH_FORWARD)

(change it to this)
if (sa->sin_port && (!q || dyn_dir == MATCH_FORWARD))
====



This problem is presend in FreeBSD 9-CURRENT as well.


Can somebody please merge this patch to CURRENT?


Credits for this patch goes to Vadim Goncharov 
nuclight.livejournal.com/124348.html
>How-To-Repeat:
have FreeBSD host with two NIC's connected to different networks with different GW each. Applay ipfw rules from example provided. Start some service (SSH/Apache/etc). Make a try to connect to connect to service from network behing GW that is not configured as default gateway on FreeBSD host - answer will go via second NIC/default GW. keep-state ruleset do not works.
>Fix:
applay patch

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-ipfw 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Jun 14 00:01:10 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=147720 

From: "Andrey V. Elsukov" <bu7cher@yandex.ru>
To: bug-followup@FreeBSD.org, dima_bsd@inbox.lv
Cc:  
Subject: Re: kern/147720: [ipfw] ipfw dynamic rules and fwd
Date: Sun, 29 May 2011 14:41:03 +0400

 This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
 --------------enig6179B1AC85A77AA253EA07DD
 Content-Type: text/plain; charset=KOI8-R
 Content-Transfer-Encoding: quoted-printable
 
 Hi,
 
 are you sure that this patch works? Do you have working configuration?
 
 --=20
 WBR, Andrey V. Elsukov
 
 
 --------------enig6179B1AC85A77AA253EA07DD
 Content-Type: application/pgp-signature; name="signature.asc"
 Content-Description: OpenPGP digital signature
 Content-Disposition: attachment; filename="signature.asc"
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.17 (FreeBSD)
 
 iQEcBAEBAgAGBQJN4iK/AAoJEAHF6gQQyKF62RwH/iuvANPzSdAvCSTKeNtC5jv3
 eBPgjbubEhv3/MMppzyd0FNM1/uRq9Rzk5XflozChhVOn00MXLc43TJ0Ow2wBUYJ
 K85rUnbeFnjmoEipXnqwtF+z8wj6YMKFM26k83MTm01IIZpN7N813AH3RC0OKyVk
 ktgXHNcf0Kx+E/GTngIIr+HHRXpmPgFd30unz6QaoE0UNDwvgA/eJT2x4sJw9GoW
 ivCTFkTgPxEpUJ2YMhh6d4rK3tx3/cLUTOOPshcDoqhY+kwm9EyiVBmZm2xui9Eu
 pV5BgIQfi+JM8pxVso5cuuT42ri8OWrSMPyuB2q5EnTcqF4UriucFz+UnfGQFfA=
 =+F6w
 -----END PGP SIGNATURE-----
 
 --------------enig6179B1AC85A77AA253EA07DD--

From: "Andrey V. Elsukov" <ae@FreeBSD.org>
To: bug-followup@FreeBSD.org, dima_bsd@inbox.lv
Cc:  
Subject: Re: kern/147720: [ipfw] ipfw dynamic rules and fwd
Date: Mon, 30 May 2011 15:37:52 +0400

 Hi,
 
 Can you test the following patch?
 http://people.freebsd.org/~ae/ipfw_fwd.diff
 
 -- 
 WBR, Andrey V. Elsukov

From: "skeletor@lissyara.su" <skeletor@lissyara.su>
To: bug-followup@FreeBSD.org, dima_bsd@inbox.lv
Cc:  
Subject: Re: kern/147720: [ipfw] ipfw dynamic rules and fwd
Date: Tue, 31 May 2011 12:03:53 +0300

 Hello!
 I have tested the NEW following code on FreeBSD 8.2 Release i386 and it 
 works!
 
 New CODE (patched)
 
 ...
 case O_FORWARD_IP: {
 +	struct sockaddr_in *sa;
 +	sa = &(((ipfw_insn_sa *)cmd)->sa);
 	if (args->eh)<->/* not valid on layer2 pkts */
 		break;
 +	if (!q || dyn_dir == MATCH_FORWARD || sa->sin_port == 0) {
 -	if (!q || dyn_dir == MATCH_FORWARD) {
 -	struct sockaddr_in *sa;
 -	sa = &(((ipfw_insn_sa *)cmd)->sa);
 ...
 
 
 This code was patched by Vadim Goncharov and tested by me (skeletor).
 

From: "skeletor@lissyara.su" <skeletor@lissyara.su>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/147720: [ipfw] ipfw dynamic rules and fwd
Date: Tue, 31 May 2011 14:01:17 +0300

 Hello!
 I have tested patch by Andrey V. Elsukov 
 (http://people.freebsd.org/~ae/ipfw_fwd.diff) and it works under FreeBSD 
 8.2 i386, kernel GENERIC.

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/147720: commit references a PR
Date: Wed,  1 Jun 2011 19:45:02 +0000 (UTC)

 Author: ae
 Date: Wed Jun  1 19:44:52 2011
 New Revision: 222582
 URL: http://svn.freebsd.org/changeset/base/222582
 
 Log:
   O_FORWARD_IP is only action which depends from the result of lookup of
   dynamic rules. We are doing forwarding in the following cases:
    o For the simple ipfw fwd rule, e.g.
   
   	fwd 10.0.0.1 ip from any to any out xmit em0
   	fwd 127.0.0.1,3128 tcp from any to any 80 in recv em1
   
    o For the dynamic fwd rule, e.g.
   
    	fwd 192.168.0.1 tcp from any to 10.0.0.3 3333 setup keep-state
   
           When this rule triggers it creates a dynamic rule, but this
   	dynamic rule should forward packets only in forward direction.
   
    o And the last case that does not work before - simple fwd rule which
    triggers when some dynamic rule is already executed.
   
   PR:		kern/147720, kern/150798
   MFC after:	1 month
 
 Modified:
   head/sys/netinet/ipfw/ip_fw2.c
 
 Modified: head/sys/netinet/ipfw/ip_fw2.c
 ==============================================================================
 --- head/sys/netinet/ipfw/ip_fw2.c	Wed Jun  1 18:42:44 2011	(r222581)
 +++ head/sys/netinet/ipfw/ip_fw2.c	Wed Jun  1 19:44:52 2011	(r222582)
 @@ -2118,7 +2118,8 @@ do {								\
  			case O_FORWARD_IP:
  				if (args->eh)	/* not valid on layer2 pkts */
  					break;
 -				if (!q || dyn_dir == MATCH_FORWARD) {
 +				if (q == NULL || q->rule != f ||
 +				    dyn_dir == MATCH_FORWARD) {
  				    struct sockaddr_in *sa;
  				    sa = &(((ipfw_insn_sa *)cmd)->sa);
  				    if (sa->sin_addr.s_addr == INADDR_ANY) {
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: "skeletor@lissyara.su" <skeletor@lissyara.su>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/147720: [ipfw] ipfw dynamic rules and fwd
Date: Tue, 21 Jun 2011 10:04:41 +0300

 Hello!
 I tested patch-1.diff and found several problems. When I use 2 channels 
 my VPN (I use mpd with connect type pptp) stop working. This problem 
 appears not on all servers.
 
 Here my results of tests:
 
 1) FreeBSD 8.1 amd64 (VPN server), 2 external real IPs - doesn't work VPN
 2) FreeBSD 8.2 i386 , 1 external real IP (second - doesn't real) - 
 doesn't work connect on second (not real) IP
 3) FreeBSD 8.1 i386 (VPN client), 2 external real IPs - all works fine
 4) FreeBSD 8.2 i386 (VPN client), 1 external real IP (second - doesn't 
 real) - connect from 2 external IPs works, but doesn't work VPN.
State-Changed-From-To: open->patched 
State-Changed-By: ae 
State-Changed-When: Sat Jul 2 08:48:17 UTC 2011 
State-Changed-Why:  
Patched in head/. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=147720 
State-Changed-From-To: patched->closed 
State-Changed-By: ae 
State-Changed-When: Wed Jul 6 06:58:55 UTC 2011 
State-Changed-Why:  
Merged to stable/7 and stable/8. Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=147720 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/147720: commit references a PR
Date: Wed,  6 Jul 2011 06:56:41 +0000 (UTC)

 Author: ae
 Date: Wed Jul  6 06:56:31 2011
 New Revision: 223819
 URL: http://svn.freebsd.org/changeset/base/223819
 
 Log:
   MFC r222582:
     O_FORWARD_IP is only action which depends from the result of lookup of
     dynamic rules. We are doing forwarding in the following cases:
      o For the simple ipfw fwd rule, e.g.
   
     	fwd 10.0.0.1 ip from any to any out xmit em0
     	fwd 127.0.0.1,3128 tcp from any to any 80 in recv em1
   
      o For the dynamic fwd rule, e.g.
   
      	fwd 192.168.0.1 tcp from any to 10.0.0.3 3333 setup keep-state
   
             When this rule triggers it creates a dynamic rule, but this
     	dynamic rule should forward packets only in forward direction.
   
      o And the last case that does not work before - simple fwd rule which
      triggers when some dynamic rule is already executed.
   
     PR:		kern/136695, kern/147720, kern/150798
 
 Modified:
   stable/8/sys/netinet/ipfw/ip_fw2.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
 
 Modified: stable/8/sys/netinet/ipfw/ip_fw2.c
 ==============================================================================
 --- stable/8/sys/netinet/ipfw/ip_fw2.c	Wed Jul  6 06:34:08 2011	(r223818)
 +++ stable/8/sys/netinet/ipfw/ip_fw2.c	Wed Jul  6 06:56:31 2011	(r223819)
 @@ -2070,7 +2070,8 @@ do {								\
  			case O_FORWARD_IP:
  				if (args->eh)	/* not valid on layer2 pkts */
  					break;
 -				if (!q || dyn_dir == MATCH_FORWARD) {
 +				if (q == NULL || q->rule != f ||
 +				    dyn_dir == MATCH_FORWARD) {
  				    struct sockaddr_in *sa;
  				    sa = &(((ipfw_insn_sa *)cmd)->sa);
  				    if (sa->sin_addr.s_addr == INADDR_ANY) {
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/147720: commit references a PR
Date: Wed,  6 Jul 2011 06:57:17 +0000 (UTC)

 Author: ae
 Date: Wed Jul  6 06:57:07 2011
 New Revision: 223820
 URL: http://svn.freebsd.org/changeset/base/223820
 
 Log:
   MFC r222582:
     O_FORWARD_IP is only action which depends from the result of lookup of
     dynamic rules. We are doing forwarding in the following cases:
      o For the simple ipfw fwd rule, e.g.
   
           fwd 10.0.0.1 ip from any to any out xmit em0
           fwd 127.0.0.1,3128 tcp from any to any 80 in recv em1
   
      o For the dynamic fwd rule, e.g.
   
           fwd 192.168.0.1 tcp from any to 10.0.0.3 3333 setup keep-state
   
             When this rule triggers it creates a dynamic rule, but this
           dynamic rule should forward packets only in forward direction.
   
      o And the last case that does not work before - simple fwd rule which
      triggers when some dynamic rule is already executed.
   
     PR:           kern/136695, kern/147720, kern/150798
 
 Modified:
   stable/7/sys/netinet/ip_fw2.c
 Directory Properties:
   stable/7/sys/   (props changed)
   stable/7/sys/cddl/contrib/opensolaris/   (props changed)
   stable/7/sys/contrib/dev/acpica/   (props changed)
   stable/7/sys/contrib/pf/   (props changed)
 
 Modified: stable/7/sys/netinet/ip_fw2.c
 ==============================================================================
 --- stable/7/sys/netinet/ip_fw2.c	Wed Jul  6 06:56:31 2011	(r223819)
 +++ stable/7/sys/netinet/ip_fw2.c	Wed Jul  6 06:57:07 2011	(r223820)
 @@ -3284,7 +3284,8 @@ check_body:
  				sa = &(((ipfw_insn_sa *)cmd)->sa);
  				if (args->eh)	/* not valid on layer2 pkts */
  					break;
 -				if (!q || dyn_dir == MATCH_FORWARD) {
 +				if (q == NULL || q->rule != f ||
 +				    dyn_dir == MATCH_FORWARD) {
  					if (sa->sin_addr.s_addr == INADDR_ANY) {
  						bcopy(sa, &args->hopstore,
  							sizeof(*sa));
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: Vadim Goncharov <vadim_nuclight@mail.ru>
To: "skeletor@lissyara.su" <skeletor@lissyara.su>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/147720: [ipfw] ipfw dynamic rules and fwd
Date: Tue, 12 Jul 2011 22:45:47 +0700

 Hi skeletor@lissyara.su! 
 
 On Tue, 21 Jun 2011 07:10:07 GMT; skeletor@lissyara.su <skeletor@lissyara.su> wrote:
 
 >  I tested patch-1.diff and found several problems. When I use 2 channels 
 >  my VPN (I use mpd with connect type pptp) stop working. This problem 
 >  appears not on all servers.
 >  
 >  Here my results of tests:
 >  
 >  1) FreeBSD 8.1 amd64 (VPN server), 2 external real IPs - doesn't work VPN
 >  2) FreeBSD 8.2 i386 , 1 external real IP (second - doesn't real) - 
 >  doesn't work connect on second (not real) IP
 >  3) FreeBSD 8.1 i386 (VPN client), 2 external real IPs - all works fine
 >  4) FreeBSD 8.2 i386 (VPN client), 1 external real IP (second - doesn't 
 >  real) - connect from 2 external IPs works, but doesn't work VPN.
 
 This is not really problem with the patch, as PPTP is using not only TCP
 connection, but also establish a GRE tunnel, independent from that TCP
 connection from the dynamic rules' point of view. There must be something
 tracking packet data payload (e.g. libalias-based NAT engine supports this)
 which will link two connections together.
 
 This message, still, does not provide any useful information even to conclude
 if there some regression with this patch. Personally I think this is the
 architectural problem with PPTP, and patch was just used in a non-appropriate
 conditions, i.e. such configuration should be avoided, and patch itself is OK.
 
 -- 
 WBR, Vadim Goncharov. ICQ#166852181       mailto:vadim_nuclight@mail.ru
 [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight]
>Unformatted:
