From pkern@cns.utoronto.ca  Mon Sep 22 10:18:26 2003
Return-Path: <pkern@cns.utoronto.ca>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 27BB716A4B3
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 22 Sep 2003 10:18:26 -0700 (PDT)
Received: from rodent.utcs.utoronto.ca (rodent.utcs.utoronto.ca [128.100.102.5])
	by mx1.FreeBSD.org (Postfix) with SMTP id 1833E43FDD
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 22 Sep 2003 10:18:25 -0700 (PDT)
	(envelope-from pkern@cns.utoronto.ca)
Received: by rodent.utcs.utoronto.ca id <1650488>; Mon, 22 Sep 2003 13:18:12 -0400
Message-Id: <03Sep22.131812edt.1650488@rodent.utcs.utoronto.ca>
Date: Mon, 22 Sep 2003 13:18:12 -0400
From: pak@cns.utoronto.ca
Reply-To: pak@cns.utoronto.ca
To: FreeBSD-gnats-submit@freebsd.org
Cc: pak@cns.utoronto.ca
Subject: disable hardware checksums when using bridge(4).
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         57100
>Category:       kern
>Synopsis:       [bridge] disable hardware checksums when using bridge(4).
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bms
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Mon Sep 22 10:20:10 PDT 2003
>Closed-Date:    Sat Sep 23 09:41:02 GMT 2006
>Last-Modified:  Sat Sep 23 19:30:16 GMT 2006
>Originator:     pak
>Release:        FreeBSD 4.8-RELEASE i386
>Organization:
Computing and Network Services, U Toronto
>Environment:
System: FreeBSD host1 4.8-RELEASE FreeBSD 4.8-RELEASE #2: Tue Sep 16 14:18:09 EDT 2003 pak@host1:/usr/src/sys/compile/BOX i386

  - dmesg excerpts:
	   ...
	xl0: <3Com 3c905B-TX Fast Etherlink XL> port [...]
	   ...
	em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.16> port [...]
	   ...
	BRIDGE 020214 loaded
	   ...

  host1# ifconfig
  xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
          options=3<rxcsum,txcsum>
          inet 300.2.2.2 netmask 0xffffff00 broadcast 300.2.2.255
		[...]
  em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
          options=3<rxcsum,txcsum>
          inet 10.1.1.1 netmask 0xfffffff8 broadcast 10.1.1.7
		[...]

  host1# sysctl -w net.link.ether.bridge=1

  host1# sysctl -w net.link.ether.bridge_cfg="xl0,em0"

  - xl0 is directly connected to the "outside world."
  - xl0 is configured with a proper IP address (eg. 300.2.2.2 ).

  - em0 is configured with a non-routable RFC-1918 address (say 10.1.1.1).
  - em0 is connected to a switch along with another system called inner1
    so that traffic to and/or from inner1 is being bridged by host1.

  - a rough diagram:
	                 :-----------:
	{  the  }        :   host1   :      |
	{outside} ----- xl0         em0 ----+
	{ world }        :           :      |
	                 :-----------:      |    :--------:
	                                    +--- : inner1 :
	                                    |    :--------:

>Description:

  When bridging is enabled, hardware checksumming by the NICs corrupts
  the checksums for packets which are generated by the bridging system
  and intended for systems "on the inside".  In the example, with
  bridging enabled, inner1 is able to talk to the outside world but it's
  not able to talk to host1 via host1's "proper" address. IP packets 
  generated by host1 and intended for inner1 have incorrect checksums.
  Disabling hardware checksums during bridging appears to fix this.


>How-To-Repeat:
>Fix:

  Included below is a patch to disable (restore) the use of hardware
  checksums on bridging NICs when bridging is enabled (disabled).
  The same sort of manipulation of 'hwassist' can be found
  in /sys/netgraph/ng_ether.c.  Hope this helps.  pak.


*** net/bridge.c	2003/09/22 15:09:35	1.1
--- net/bridge.c	2003/09/22 14:51:23
***************
*** 314,319 ****
--- 314,320 ----
  
  	if ( b->flags & IFF_BDG_PROMISC ) {
  	    s = splimp();
+ 	    ifp->if_hwassist = b->hwassist;	/* restore hwassist. */
  	    ifpromisc(ifp, 0);
  	    splx(s);
  	    b->flags &= ~(IFF_BDG_PROMISC|IFF_MUTE) ;
***************
*** 361,366 ****
--- 362,369 ----
  	if ( !(b->flags & IFF_BDG_PROMISC) ) {
  	    int ret ;
  	    s = splimp();
+ 	    b->hwassist = ifp->if_hwassist; 	/* save hwassist. */
+ 	    ifp->if_hwassist = 0;		/* disable hwassist. */
  	    ret = ifpromisc(ifp, 1);
  	    splx(s);
  	    b->flags |= IFF_BDG_PROMISC ;
*** net/bridge.h	2003/09/22 14:52:05	1.1
--- net/bridge.h	2003/09/22 14:53:12
***************
*** 47,52 ****
--- 47,53 ----
  #define IFF_BDG_PROMISC 0x0001  /* set promisc mode on this if.	*/
  #define IFF_MUTE        0x0002  /* mute this if for bridging.   */
  #define IFF_USED        0x0004  /* use this if for bridging.    */
+     u_long hwassist;		/* hardware checksum capabilities */
      struct cluster_softc *cluster;
  } ;
  


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->bms 
Responsible-Changed-By: bms 
Responsible-Changed-When: Tue 25 Nov 2003 15:56:35 PST 
Responsible-Changed-Why:  
Another one for the evil network junta. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=57100 
State-Changed-From-To: open->analyzed 
State-Changed-By: bms 
State-Changed-When: Sat Jul 3 17:39:25 GMT 2004 
State-Changed-Why:  
Improved patch about to hit this PR and the lists 

http://www.freebsd.org/cgi/query-pr.cgi?pr=57100 

From: Bruce M Simpson <bms@spc.org>
To: pak <pak@cns.utoronto.ca>
Cc: freebsd-net@freebsd.org, freebsd-gnats-submit@freebsd.org
Subject: Re: kern/57100: disable hardware checksums when using bridge(4).
Date: Sat, 3 Jul 2004 18:39:45 +0100

 --JvUS8mwutKMHKosv
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 Here is a somewhat improved patch for tweaking the behaviour of the bridge
 code with respect to disabling the hardware-assist-checksum features of
 driver instances which are members of the bridge.
 
 I don't make use of the bridge code so I'd appreciate it if others who do
 could test this patch.
 
 To disable *all* capabilities, setting net.link.ether.bridge.hwassmask to
 0 should suffice.
 
 Regards,
 BMS
 
 --JvUS8mwutKMHKosv
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename="bridge-hwassist.patch"
 
 This patch implements a sysctl MIB variable, net.link.ether.bridge.hwassmask,
 which may be used to specify a bitmask which is applied to each member
 interface of a bridge to selectively disable interface hardware assist
 capabilities, and provides finer run-time control over such behaviour.
 
 Index: bridge.c
 ===================================================================
 RCS file: /home/ncvs/src/sys/net/bridge.c,v
 retrieving revision 1.79
 diff -u -p -r1.79 bridge.c
 --- bridge.c	15 Jun 2004 23:57:41 -0000	1.79
 +++ bridge.c	3 Jul 2004 17:33:47 -0000
 @@ -204,6 +204,10 @@ SYSCTL_NODE(_net_link_ether, OID_AUTO, b
  static char bridge_version[] = "031224";
  SYSCTL_STRING(_net_link_ether_bridge, OID_AUTO, version, CTLFLAG_RD,
  	bridge_version, 0, "software version");
 +static u_long bridge_hwassmask = 0xFFFFFFFFUL;
 +SYSCTL_ULONG(_net_link_ether_bridge, OID_AUTO, hwassmask, CTLFLAG_RW,
 +	&bridge_hwassmask, 0,
 +	"Mask to apply to if_hwassist field for bridge interfaces");
  
  #define BRIDGE_DEBUG
  #ifdef BRIDGE_DEBUG
 @@ -391,6 +395,7 @@ bridge_off(void)
  
  	if ( b->flags & IFF_BDG_PROMISC ) {
  	    ifpromisc(ifp, 0);
 +	    ifp->if_hwassist = b->hwassist;
  	    b->flags &= ~(IFF_BDG_PROMISC|IFF_MUTE) ;
  	    DPRINTF(("%s: %s promisc OFF if_flags 0x%x "
  		"bdg_flags 0x%x\n", __func__, ifp->if_xname,
 @@ -433,6 +438,8 @@ bridge_on(void)
  	    if_up(ifp);
  	}
  	if ( !(b->flags & IFF_BDG_PROMISC) ) {
 +	    b->hwassist = ifp->if_hwassist;
 +	    ifp->if_hwassist &= bridge_hwassmask;
  	    (void) ifpromisc(ifp, 1);
  	    b->flags |= IFF_BDG_PROMISC ;
  	    DPRINTF(("%s: %s promisc ON if_flags 0x%x bdg_flags 0x%x\n",
 Index: bridge.h
 ===================================================================
 RCS file: /home/ncvs/src/sys/net/bridge.h,v
 retrieving revision 1.12
 diff -u -p -r1.12 bridge.h
 --- bridge.h	15 Nov 2002 00:00:14 -0000	1.12
 +++ bridge.h	3 Jul 2004 17:26:44 -0000
 @@ -48,6 +48,7 @@ struct bdg_softc {
  #define IFF_MUTE        0x0002  /* mute this if for bridging.   */
  #define IFF_USED        0x0004  /* use this if for bridging.    */
      struct cluster_softc *cluster;
 +    u_long hwassist;		/* saved ifp->if_hwassist field */
  } ;
  
  extern struct bdg_softc *ifp2sc;
 
 --JvUS8mwutKMHKosv--
State-Changed-From-To: analyzed->closed 
State-Changed-By: bms 
State-Changed-When: Sat Sep 23 09:40:26 UTC 2006 
State-Changed-Why:  
I committed my patch on RELENG_4. 
In HEAD, the old bridge code was removed in favour of if_bridge 
which is smarter about such things. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=57100 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/57100: commit references a PR
Date: Sat, 23 Sep 2006 09:40:29 +0000 (UTC)

 bms         2006-09-23 09:40:02 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_4)
     sys/net              bridge.h 
   Log:
   Add a sysctl net.link.ether.bridge.hwassmask which lets administrators
   mask hardware checksums on bridge interface members.
   
   PR:             kern/57100
   
   Revision  Changes    Path
   1.4.2.6   +1 -0      src/sys/net/bridge.h
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/57100: commit references a PR
Date: Sat, 23 Sep 2006 19:27:55 +0000 (UTC)

 bms         2006-09-23 19:27:40 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_4)
     sys/net              bridge.c 
   Log:
   Add a sysctl net.link.ether.bridge.hwassmask which lets administrators
   mask hardware checksums on bridge interface members.
   
   PR:             kern/57100
   
   Revision   Changes    Path
   1.16.2.26  +10 -0     src/sys/net/bridge.c
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
>Unformatted:
