From nobody@FreeBSD.org  Sun Feb 26 03:18:03 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 34E8B16A420
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 26 Feb 2006 03:18:03 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id D9D7643D45
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 26 Feb 2006 03:18:02 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k1Q3I2di007834
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 26 Feb 2006 03:18:02 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id k1Q3I2LB007833;
	Sun, 26 Feb 2006 03:18:02 GMT
	(envelope-from nobody)
Message-Id: <200602260318.k1Q3I2LB007833@www.freebsd.org>
Date: Sun, 26 Feb 2006 03:18:02 GMT
From: Adam McDougall <mcdouga9@egr.msu.edu>
To: freebsd-gnats-submit@FreeBSD.org
Subject: pf no-df breaks IP checksum of all tcp traffic through if_bridge
X-Send-Pr-Version: www-2.3

>Number:         93849
>Category:       kern
>Synopsis:       pf no-df breaks IP checksum of all tcp traffic through if_bridge
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-pf
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 26 03:20:04 GMT 2006
>Closed-Date:    Tue Mar 28 15:10:37 GMT 2006
>Last-Modified:  Tue Mar 28 15:10:37 GMT 2006
>Originator:     Adam McDougall
>Release:        FreeBSD 6.1-PRERELEASE #5: Wed Feb 22 14:55:45 EST 2006
>Organization:
>Environment:
FreeBSD fw1 6.1-PRERELEASE FreeBSD 6.1-PRERELEASE #5: Wed Feb 22 14:55:45 EST 2006     user@fw1:/usr/obj/usr/src/sys/TYAN_GS12  i386

>Description:
I have setup if_bridge and pf on a server with dual em interfaces
running FreeBSD 6.1-PRERELEASE #5: Wed Feb 22 14:55:45 EST 2006.

rc.conf relevant items: (The IP's are just for temporary management from
either side of the firewall as needed)
ifconfig_em0="inet 10.0.0.80 netmask 0xffffff00"
ifconfig_em0_alias0="inet 35.9.44.100 netmask 0xffffff00"
ifconfig_em1="inet 10.0.1.80 netmask 0xffffff00"
cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0 addm em1 up"

I have narrowed my ruleset down to a simple config for testing:

ext_if="em0"
int_if="em1"
scrub in on $ext_if no-df
pass in all
pass out all
pass quick on lo0

# pfctl -Rf /etc/pf.conf
No ALTQ support in kernel
ALTQ related functions disabled

# pfctl -sr
No ALTQ support in kernel
ALTQ related functions disabled
scrub in on em0 all no-df fragment reassemble
pass in all
pass out all
pass quick on lo0 all

Whenever I have no-df in the scrub line, the bridging firewall still
passes my ssh SYN packet to the host behind the firewall, but the
receiving host discards it due to a bad IP checksum (I believe).

Using tcpdump on em0 and em1 on the firewall, I see the packet come in
with DF set, and leave with DF unset however the IP checksum is reported
bad on the em1 side according to ethereal.  I verified that the IP checksum was unmodified between em0 and em1.  I also tried ifconfig -rxcsum -txcsum on both nics but no improvement in behavior.  Running tcpdump on the receiving host shows the SYN packet, but trying to use -w to save it
to a file results in no packets captured.  All systems involved are FreeBSD
so far, and the symptoms persist going both directions across the bridge.
ping still works.

I am trying to get no-df to work because documentation indicates it is
needed to pass NFS which will be a requirement for me.  I didn't get
very far with attempting to exclude just NFS traffic from being scrubbed,
but it seems to be that a firewall munging packets ought to produce ones
with valid checksums.  Please let me know if I need to provide more information
or what else I can do to debug this further.

>How-To-Repeat:
Setup an if_bridge between two interfaces on freebsd, add a scrub no-df rule in pf, witness resulting tcp packets get dropped by the receiving host kernel. 
>Fix:
              
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-pf 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sun Feb 26 07:25:04 UTC 2006 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=93849 

From: Max Laier <max@love2party.net>
To: bug-followup@freebsd.org,
 mcdouga9@egr.msu.edu
Cc:  
Subject: Re: kern/93849: pf no-df breaks IP checksum of all tcp traffic through if_bridge
Date: Sat, 4 Mar 2006 16:04:29 +0100

 --Boundary-00=_/xaCEsIJLMyzIMC
 Content-Type: text/plain;
   charset="us-ascii"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline
 
 Please try the attached patch.
 -- 
   Max
 
 --Boundary-00=_/xaCEsIJLMyzIMC
 Content-Type: text/x-diff;
   charset="us-ascii";
   name="nodf.fix.diff"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename="nodf.fix.diff"
 
 Index: pf_norm.c
 ===================================================================
 RCS file: /usr/store/mlaier/fcvs/src/sys/contrib/pf/net/pf_norm.c,v
 retrieving revision 1.16
 diff -u -r1.16 pf_norm.c
 --- pf_norm.c	19 Jan 2006 11:46:45 -0000	1.16
 +++ pf_norm.c	4 Mar 2006 14:49:13 -0000
 @@ -988,8 +988,12 @@
  		goto drop;
  
  	/* Clear IP_DF if the rule uses the no-df option */
 -	if (r->rule_flag & PFRULE_NODF)
 +	if ((r->rule_flag & PFRULE_NODF) {
 +		u_int16_t old = h->ip_off;
 +
  		h->ip_off &= htons(~IP_DF);
 +		h->ip_sum = pf_cksum_fixup(h->ip_sum, old, h->ip_off, 0);
 +	}
  
  	/* We will need other tests here */
  	if (!fragoff && !mff)
 
 --Boundary-00=_/xaCEsIJLMyzIMC--
State-Changed-From-To: open->patched 
State-Changed-By: mlaier 
State-Changed-When: Sat Mar 25 21:15:51 UTC 2006 
State-Changed-Why:  
A more complete patch has been committed.  Thanks for the report.  MFC due 
in three days. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=93849 
State-Changed-From-To: patched->closed 
State-Changed-By: mlaier 
State-Changed-When: Tue Mar 28 15:09:36 UTC 2006 
State-Changed-Why:  
MFCed to RELENG_5 and RELENG_6. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=93849 
>Unformatted:
