From nobody@FreeBSD.org  Thu Oct 26 10:22:31 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id EBD3A16A40F
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 26 Oct 2006 10:22:30 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id B703943D4C
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 26 Oct 2006 10:22:30 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k9QAMUQo022549
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 26 Oct 2006 10:22:30 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id k9QAMUdt022544;
	Thu, 26 Oct 2006 10:22:30 GMT
	(envelope-from nobody)
Message-Id: <200610261022.k9QAMUdt022544@www.freebsd.org>
Date: Thu, 26 Oct 2006 10:22:30 GMT
From: Frans van der Veer<Frans.vanderVeer@peektraffic.nl>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD 6.1 not RFC 768 (UDP) compliant on Compaq/HP DL-380G3
X-Send-Pr-Version: www-3.0

>Number:         104826
>Category:       kern
>Synopsis:       [bge] FreeBSD 6.1 not RFC 768 (UDP) compliant on Compaq/HP DL-380G3
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    yongari
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Oct 26 10:30:15 GMT 2006
>Closed-Date:    Fri Sep 24 19:49:44 UTC 2010
>Last-Modified:  Fri Sep 24 19:49:44 UTC 2010
>Originator:     Frans van der Veer
>Release:        6.1
>Organization:
PEEK Traffic BV, The Netherlands
>Environment:
FreeBSD gandalf.rd.nl.peeknet.prv. 6.1-RELEASE FreeBSD 6.1-RELEASE #1: Wed Oct 25 11:31:21 CEST 2006     root@gandalf.rd.nl.peeknet.prv.:/usr/src/sys/i386/compile/GANDALF  i386
>Description:
During testing of a Profibus/Ethernet gateway device that was suspected not have implemented UDP RFC 768 correctly, we stumbled onto a possible bug on FreeBSD 6.1 in the same area.

It appears that 6.1 is not capable of sending out UDP checksum 0xffff when the internal combination of pseudo-header, upd header and udp payload requires that.
In stead of that, the machine sends out 0x0000 as UDP checksum which is interpreted as "no checksum calculated".

We tracked the failure down to the bge driver. We found that the bge device driver uses a feature called "checksum offload". When we switch that feature off, the system operates accoring to RFC 768.

We noticed this with FreeBSD 6.1 on a Compaq/HP DL380G3 system.
Relevant lines from the messages file are:

Oct 25 11:51:55 gandalf kernel: FreeBSD 6.1-RELEASE #1: Wed Oct 25 11:31:21 CEST 2006
-
-
Oct 25 11:51:55 gandalf kernel: bge0: <Broadcom BCM5703 Gigabit Ethernet, ASIC rev. 0x1002> mem 0xf06f0000-0xf06fffff irq 29 at device 1.0 on pci2
Oct 25 11:51:55 gandalf kernel: bge0: No checksum offload

(This log is from after we created a patch for ourselves, the last log line is due to that patch)

We created a dirty fix to get outselves out of trouble for the moment: we switched of the "hwassist" feature in the initialisation phase of the bge driver to give the checksum calculation task back to software in stead of the Broadcom hardware.

Kind regards,

Frans van der Veer
Senior software engineer.
R&D department,
PEEK Traffic B.V., The Netherlands.
>How-To-Repeat:
We repeated the problem by creating a small UDP echo program that answered on a machine sending UDP packets. With ethereal (run on another machine since ethereal is troubles by the same checksum offload feature as well) we checked the checksum, did a hexadecimal calculator exercise and modified the data the echo program is sending out to achieve checksum 0xffff.
>Fix:
As told, we created a "dirty fix" to get us out of trouble for the time being. This is how:

Patched File /usr/src/sys/dev/if_bge.c: (from line 2213 on)

/*
	 * 5700 B0 chips do not support checksumming correctly due
	 * to hardware bugs.
	 */
	if ( 1 ) {
		ifp->if_capabilities &= ~IFCAP_HWCSUM;
		ifp->if_capenable &= IFCAP_HWCSUM;
		ifp->if_hwassist = 0;
		device_printf(sc->bge_dev, "No checksum offload\n");

	}

Original code:

      /*
	 * 5700 B0 chips do not support checksumming correctly due
	 * to hardware bugs.
	 */
	if (sc->bge_chipid == BGE_CHIPID_BCM5700_B0) {
		ifp->if_capabilities &= ~IFCAP_HWCSUM;
		ifp->if_capenable &= IFCAP_HWCSUM;
		ifp->if_hwassist = 0;
	}As told, we created a "dirty fix" to get us out of trouble for the time being. See attached documents on how we did that.
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-i386->andre 
Responsible-Changed-By: remko 
Responsible-Changed-When: Sat Nov 11 13:42:49 UTC 2006 
Responsible-Changed-Why:  
Assign to Andre, thanks andre! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=104826 
Responsible-Changed-From-To: andre->yongari 
Responsible-Changed-By: andre 
Responsible-Changed-When: Tue Aug 17 20:09:38 UTC 2010 
Responsible-Changed-Why:  
Send over to yongari. He's our driver expert and this is not a 
network stack bug but some hardware or driver issue. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=104826 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/104826: commit references a PR
Date: Sun, 22 Aug 2010 01:39:20 +0000 (UTC)

 Author: yongari
 Date: Sun Aug 22 01:39:09 2010
 New Revision: 211596
 URL: http://svn.freebsd.org/changeset/base/211596
 
 Log:
   It seems all Broadcom controllers have a bug that can generate UDP
   datagrams with checksum value 0 when TX UDP checksum offloading is
   enabled.  Generating UDP checksum value 0 is RFC 768 violation.
   Even though the probability of generating such UDP datagrams is
   low, I don't want to see FreeBSD boxes to inject such datagrams
   into network so disable UDP checksum offloading by default.  Users
   still override this behavior by setting a sysctl variable or loader
   tunable, dev.bge.%d.forced_udpcsum.
   
   I have no idea why this issue was not reported so far given that
   bge(4) is one of the most commonly used controller on high-end
   server class systems. Thanks to andre@ who passed the PR to me.
   
   PR:	kern/104826
 
 Modified:
   head/sys/dev/bge/if_bge.c
   head/sys/dev/bge/if_bgereg.h
 
 Modified: head/sys/dev/bge/if_bge.c
 ==============================================================================
 --- head/sys/dev/bge/if_bge.c	Sun Aug 22 00:04:24 2010	(r211595)
 +++ head/sys/dev/bge/if_bge.c	Sun Aug 22 01:39:09 2010	(r211596)
 @@ -120,7 +120,7 @@ __FBSDID("$FreeBSD$");
  
  #include <dev/bge/if_bgereg.h>
  
 -#define	BGE_CSUM_FEATURES	(CSUM_IP | CSUM_TCP | CSUM_UDP)
 +#define	BGE_CSUM_FEATURES	(CSUM_IP | CSUM_TCP)
  #define	ETHER_MIN_NOPAD		(ETHER_MIN_LEN - ETHER_CRC_LEN) /* i.e., 60 */
  
  MODULE_DEPEND(bge, pci, 1, 1, 1);
 @@ -2795,6 +2795,8 @@ bge_attach(device_t dev)
  		goto fail;
  	}
  
 +	bge_add_sysctls(sc);
 +
  	/* Set default tuneable values. */
  	sc->bge_stat_ticks = BGE_TICKS_PER_SEC;
  	sc->bge_rx_coal_ticks = 150;
 @@ -2802,6 +2804,11 @@ bge_attach(device_t dev)
  	sc->bge_rx_max_coal_bds = 10;
  	sc->bge_tx_max_coal_bds = 10;
  
 +	/* Initialize checksum features to use. */
 +	sc->bge_csum_features = BGE_CSUM_FEATURES;
 +	if (sc->bge_forced_udpcsum != 0)
 +		sc->bge_csum_features |= CSUM_UDP;
 +
  	/* Set up ifnet structure */
  	ifp = sc->bge_ifp = if_alloc(IFT_ETHER);
  	if (ifp == NULL) {
 @@ -2818,7 +2825,7 @@ bge_attach(device_t dev)
  	ifp->if_snd.ifq_drv_maxlen = BGE_TX_RING_CNT - 1;
  	IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen);
  	IFQ_SET_READY(&ifp->if_snd);
 -	ifp->if_hwassist = BGE_CSUM_FEATURES;
 +	ifp->if_hwassist = sc->bge_csum_features;
  	ifp->if_capabilities = IFCAP_HWCSUM | IFCAP_VLAN_HWTAGGING |
  	    IFCAP_VLAN_MTU;
  	if ((sc->bge_flags & BGE_FLAG_TSO) != 0) {
 @@ -2975,8 +2982,6 @@ again:
  		device_printf(sc->bge_dev, "couldn't set up irq\n");
  	}
  
 -	bge_add_sysctls(sc);
 -
  	return (0);
  
  fail:
 @@ -3960,7 +3965,7 @@ bge_encap(struct bge_softc *sc, struct m
  			return (ENOBUFS);
  		csum_flags |= BGE_TXBDFLAG_CPU_PRE_DMA |
  		    BGE_TXBDFLAG_CPU_POST_DMA;
 -	} else if ((m->m_pkthdr.csum_flags & BGE_CSUM_FEATURES) != 0) {
 +	} else if ((m->m_pkthdr.csum_flags & sc->bge_csum_features) != 0) {
  		if (m->m_pkthdr.csum_flags & CSUM_IP)
  			csum_flags |= BGE_TXBDFLAG_IP_CSUM;
  		if (m->m_pkthdr.csum_flags & (CSUM_TCP | CSUM_UDP)) {
 @@ -4237,6 +4242,17 @@ bge_init_locked(struct bge_softc *sc)
  	/* Program VLAN tag stripping. */
  	bge_setvlan(sc);
  
 +	/* Override UDP checksum offloading. */
 +	if (sc->bge_forced_udpcsum == 0)
 +		sc->bge_csum_features &= ~CSUM_UDP;
 +	else
 +		sc->bge_csum_features |= CSUM_UDP;
 +	if (ifp->if_capabilities & IFCAP_TXCSUM &&
 +	    ifp->if_capenable & IFCAP_TXCSUM) {
 +		ifp->if_hwassist &= ~(BGE_CSUM_FEATURES | CSUM_UDP);
 +		ifp->if_hwassist |= sc->bge_csum_features;
 +	}
 +
  	/* Init RX ring. */
  	if (bge_init_rx_ring_std(sc) != 0) {
  		device_printf(sc->bge_dev, "no memory for std Rx buffers.\n");
 @@ -4562,9 +4578,9 @@ bge_ioctl(struct ifnet *ifp, u_long comm
  			ifp->if_capenable ^= IFCAP_HWCSUM;
  			if (IFCAP_HWCSUM & ifp->if_capenable &&
  			    IFCAP_HWCSUM & ifp->if_capabilities)
 -				ifp->if_hwassist |= BGE_CSUM_FEATURES;
 +				ifp->if_hwassist |= sc->bge_csum_features;
  			else
 -				ifp->if_hwassist &= ~BGE_CSUM_FEATURES;
 +				ifp->if_hwassist &= ~sc->bge_csum_features;
  		}
  
  		if ((mask & IFCAP_TSO4) != 0 &&
 @@ -4940,6 +4956,24 @@ bge_add_sysctls(struct bge_softc *sc)
  	    "Number of fragmented TX buffers of a frame allowed before "
  	    "forced collapsing");
  
 +	/*
 +	 * It seems all Broadcom controllers have a bug that can generate UDP
 +	 * datagrams with checksum value 0 when TX UDP checksum offloading is
 +	 * enabled.  Generating UDP checksum value 0 is RFC 768 violation.
 +	 * Even though the probability of generating such UDP datagrams is
 +	 * low, I don't want to see FreeBSD boxes to inject such datagrams
 +	 * into network so disable UDP checksum offloading by default.  Users
 +	 * still override this behavior by setting a sysctl variable,
 +	 * dev.bge.0.forced_udpcsum.
 +	 */
 +	sc->bge_forced_udpcsum = 0;
 +	snprintf(tn, sizeof(tn), "dev.bge.%d.bge_forced_udpcsum", unit);
 +	TUNABLE_INT_FETCH(tn, &sc->bge_forced_udpcsum);
 +	SYSCTL_ADD_INT(ctx, children, OID_AUTO, "forced_udpcsum",
 +	    CTLFLAG_RW, &sc->bge_forced_udpcsum, 0,
 +	    "Enable UDP checksum offloading even if controller can "
 +	    "generate UDP checksum value 0");
 +
  	if (BGE_IS_5705_PLUS(sc))
  		return;
  
 
 Modified: head/sys/dev/bge/if_bgereg.h
 ==============================================================================
 --- head/sys/dev/bge/if_bgereg.h	Sun Aug 22 00:04:24 2010	(r211595)
 +++ head/sys/dev/bge/if_bgereg.h	Sun Aug 22 01:39:09 2010	(r211596)
 @@ -2644,6 +2644,8 @@ struct bge_softc {
  	int			bge_link_evt;	/* pending link event */
  	int			bge_timer;
  	int			bge_forced_collapse;
 +	int			bge_forced_udpcsum;
 +	int			bge_csum_features;
  	struct callout		bge_stat_ch;
  	uint32_t		bge_rx_discards;
  	uint32_t		bge_tx_discards;
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: yongari 
State-Changed-When: Sun Aug 22 01:46:25 UTC 2010 
State-Changed-Why:  
Fix commited to HEAD(r211596). 
Thanks for reporting! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=104826 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/104826: commit references a PR
Date: Fri, 24 Sep 2010 18:00:23 +0000 (UTC)

 Author: yongari
 Date: Fri Sep 24 17:59:58 2010
 New Revision: 213109
 URL: http://svn.freebsd.org/changeset/base/213109
 
 Log:
   MFC r211596:
     It seems all Broadcom controllers have a bug that can generate UDP
     datagrams with checksum value 0 when TX UDP checksum offloading is
     enabled.  Generating UDP checksum value 0 is RFC 768 violation.
     Even though the probability of generating such UDP datagrams is
     low, I don't want to see FreeBSD boxes to inject such datagrams
     into network so disable UDP checksum offloading by default.  Users
     still override this behavior by setting a sysctl variable or loader
     tunable, dev.bge.%d.forced_udpcsum.
   
     I have no idea why this issue was not reported so far given that
     bge(4) is one of the most commonly used controller on high-end
     server class systems. Thanks to andre@ who passed the PR to me.
   
     PR:	kern/104826
 
 Modified:
   stable/8/sys/dev/bge/if_bge.c
   stable/8/sys/dev/bge/if_bgereg.h
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
   stable/8/sys/dev/xen/xenpci/   (props changed)
 
 Modified: stable/8/sys/dev/bge/if_bge.c
 ==============================================================================
 --- stable/8/sys/dev/bge/if_bge.c	Fri Sep 24 17:56:12 2010	(r213108)
 +++ stable/8/sys/dev/bge/if_bge.c	Fri Sep 24 17:59:58 2010	(r213109)
 @@ -120,7 +120,7 @@ __FBSDID("$FreeBSD$");
  
  #include <dev/bge/if_bgereg.h>
  
 -#define	BGE_CSUM_FEATURES	(CSUM_IP | CSUM_TCP | CSUM_UDP)
 +#define	BGE_CSUM_FEATURES	(CSUM_IP | CSUM_TCP)
  #define	ETHER_MIN_NOPAD		(ETHER_MIN_LEN - ETHER_CRC_LEN) /* i.e., 60 */
  
  MODULE_DEPEND(bge, pci, 1, 1, 1);
 @@ -2795,6 +2795,8 @@ bge_attach(device_t dev)
  		goto fail;
  	}
  
 +	bge_add_sysctls(sc);
 +
  	/* Set default tuneable values. */
  	sc->bge_stat_ticks = BGE_TICKS_PER_SEC;
  	sc->bge_rx_coal_ticks = 150;
 @@ -2802,6 +2804,11 @@ bge_attach(device_t dev)
  	sc->bge_rx_max_coal_bds = 10;
  	sc->bge_tx_max_coal_bds = 10;
  
 +	/* Initialize checksum features to use. */
 +	sc->bge_csum_features = BGE_CSUM_FEATURES;
 +	if (sc->bge_forced_udpcsum != 0)
 +		sc->bge_csum_features |= CSUM_UDP;
 +
  	/* Set up ifnet structure */
  	ifp = sc->bge_ifp = if_alloc(IFT_ETHER);
  	if (ifp == NULL) {
 @@ -2818,7 +2825,7 @@ bge_attach(device_t dev)
  	ifp->if_snd.ifq_drv_maxlen = BGE_TX_RING_CNT - 1;
  	IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen);
  	IFQ_SET_READY(&ifp->if_snd);
 -	ifp->if_hwassist = BGE_CSUM_FEATURES;
 +	ifp->if_hwassist = sc->bge_csum_features;
  	ifp->if_capabilities = IFCAP_HWCSUM | IFCAP_VLAN_HWTAGGING |
  	    IFCAP_VLAN_MTU;
  	if ((sc->bge_flags & BGE_FLAG_TSO) != 0) {
 @@ -2975,8 +2982,6 @@ again:
  		device_printf(sc->bge_dev, "couldn't set up irq\n");
  	}
  
 -	bge_add_sysctls(sc);
 -
  	return (0);
  
  fail:
 @@ -3960,7 +3965,7 @@ bge_encap(struct bge_softc *sc, struct m
  			return (ENOBUFS);
  		csum_flags |= BGE_TXBDFLAG_CPU_PRE_DMA |
  		    BGE_TXBDFLAG_CPU_POST_DMA;
 -	} else if ((m->m_pkthdr.csum_flags & BGE_CSUM_FEATURES) != 0) {
 +	} else if ((m->m_pkthdr.csum_flags & sc->bge_csum_features) != 0) {
  		if (m->m_pkthdr.csum_flags & CSUM_IP)
  			csum_flags |= BGE_TXBDFLAG_IP_CSUM;
  		if (m->m_pkthdr.csum_flags & (CSUM_TCP | CSUM_UDP)) {
 @@ -4237,6 +4242,17 @@ bge_init_locked(struct bge_softc *sc)
  	/* Program VLAN tag stripping. */
  	bge_setvlan(sc);
  
 +	/* Override UDP checksum offloading. */
 +	if (sc->bge_forced_udpcsum == 0)
 +		sc->bge_csum_features &= ~CSUM_UDP;
 +	else
 +		sc->bge_csum_features |= CSUM_UDP;
 +	if (ifp->if_capabilities & IFCAP_TXCSUM &&
 +	    ifp->if_capenable & IFCAP_TXCSUM) {
 +		ifp->if_hwassist &= ~(BGE_CSUM_FEATURES | CSUM_UDP);
 +		ifp->if_hwassist |= sc->bge_csum_features;
 +	}
 +
  	/* Init RX ring. */
  	if (bge_init_rx_ring_std(sc) != 0) {
  		device_printf(sc->bge_dev, "no memory for std Rx buffers.\n");
 @@ -4562,9 +4578,9 @@ bge_ioctl(struct ifnet *ifp, u_long comm
  			ifp->if_capenable ^= IFCAP_HWCSUM;
  			if (IFCAP_HWCSUM & ifp->if_capenable &&
  			    IFCAP_HWCSUM & ifp->if_capabilities)
 -				ifp->if_hwassist |= BGE_CSUM_FEATURES;
 +				ifp->if_hwassist |= sc->bge_csum_features;
  			else
 -				ifp->if_hwassist &= ~BGE_CSUM_FEATURES;
 +				ifp->if_hwassist &= ~sc->bge_csum_features;
  		}
  
  		if ((mask & IFCAP_TSO4) != 0 &&
 @@ -4940,6 +4956,24 @@ bge_add_sysctls(struct bge_softc *sc)
  	    "Number of fragmented TX buffers of a frame allowed before "
  	    "forced collapsing");
  
 +	/*
 +	 * It seems all Broadcom controllers have a bug that can generate UDP
 +	 * datagrams with checksum value 0 when TX UDP checksum offloading is
 +	 * enabled.  Generating UDP checksum value 0 is RFC 768 violation.
 +	 * Even though the probability of generating such UDP datagrams is
 +	 * low, I don't want to see FreeBSD boxes to inject such datagrams
 +	 * into network so disable UDP checksum offloading by default.  Users
 +	 * still override this behavior by setting a sysctl variable,
 +	 * dev.bge.0.forced_udpcsum.
 +	 */
 +	sc->bge_forced_udpcsum = 0;
 +	snprintf(tn, sizeof(tn), "dev.bge.%d.bge_forced_udpcsum", unit);
 +	TUNABLE_INT_FETCH(tn, &sc->bge_forced_udpcsum);
 +	SYSCTL_ADD_INT(ctx, children, OID_AUTO, "forced_udpcsum",
 +	    CTLFLAG_RW, &sc->bge_forced_udpcsum, 0,
 +	    "Enable UDP checksum offloading even if controller can "
 +	    "generate UDP checksum value 0");
 +
  	if (BGE_IS_5705_PLUS(sc))
  		return;
  
 
 Modified: stable/8/sys/dev/bge/if_bgereg.h
 ==============================================================================
 --- stable/8/sys/dev/bge/if_bgereg.h	Fri Sep 24 17:56:12 2010	(r213108)
 +++ stable/8/sys/dev/bge/if_bgereg.h	Fri Sep 24 17:59:58 2010	(r213109)
 @@ -2644,6 +2644,8 @@ struct bge_softc {
  	int			bge_link_evt;	/* pending link event */
  	int			bge_timer;
  	int			bge_forced_collapse;
 +	int			bge_forced_udpcsum;
 +	int			bge_csum_features;
  	struct callout		bge_stat_ch;
  	uint32_t		bge_rx_discards;
  	uint32_t		bge_tx_discards;
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/104826: commit references a PR
Date: Fri, 24 Sep 2010 18:01:49 +0000 (UTC)

 Author: yongari
 Date: Fri Sep 24 18:01:43 2010
 New Revision: 213110
 URL: http://svn.freebsd.org/changeset/base/213110
 
 Log:
   MFC r211596:
     It seems all Broadcom controllers have a bug that can generate UDP
     datagrams with checksum value 0 when TX UDP checksum offloading is
     enabled.  Generating UDP checksum value 0 is RFC 768 violation.
     Even though the probability of generating such UDP datagrams is
     low, I don't want to see FreeBSD boxes to inject such datagrams
     into network so disable UDP checksum offloading by default.  Users
     still override this behavior by setting a sysctl variable or loader
     tunable, dev.bge.%d.forced_udpcsum.
   
     I have no idea why this issue was not reported so far given that
     bge(4) is one of the most commonly used controller on high-end
     server class systems. Thanks to andre@ who passed the PR to me.
   
     PR:	kern/104826
 
 Modified:
   stable/7/sys/dev/bge/if_bge.c
   stable/7/sys/dev/bge/if_bgereg.h
 Directory Properties:
   stable/7/sys/   (props changed)
   stable/7/sys/cddl/contrib/opensolaris/   (props changed)
   stable/7/sys/contrib/dev/acpica/   (props changed)
   stable/7/sys/contrib/pf/   (props changed)
 
 Modified: stable/7/sys/dev/bge/if_bge.c
 ==============================================================================
 --- stable/7/sys/dev/bge/if_bge.c	Fri Sep 24 17:59:58 2010	(r213109)
 +++ stable/7/sys/dev/bge/if_bge.c	Fri Sep 24 18:01:43 2010	(r213110)
 @@ -120,7 +120,7 @@ __FBSDID("$FreeBSD$");
  
  #include <dev/bge/if_bgereg.h>
  
 -#define	BGE_CSUM_FEATURES	(CSUM_IP | CSUM_TCP | CSUM_UDP)
 +#define	BGE_CSUM_FEATURES	(CSUM_IP | CSUM_TCP)
  #define	ETHER_MIN_NOPAD		(ETHER_MIN_LEN - ETHER_CRC_LEN) /* i.e., 60 */
  
  MODULE_DEPEND(bge, pci, 1, 1, 1);
 @@ -2795,6 +2795,8 @@ bge_attach(device_t dev)
  		goto fail;
  	}
  
 +	bge_add_sysctls(sc);
 +
  	/* Set default tuneable values. */
  	sc->bge_stat_ticks = BGE_TICKS_PER_SEC;
  	sc->bge_rx_coal_ticks = 150;
 @@ -2802,6 +2804,11 @@ bge_attach(device_t dev)
  	sc->bge_rx_max_coal_bds = 10;
  	sc->bge_tx_max_coal_bds = 10;
  
 +	/* Initialize checksum features to use. */
 +	sc->bge_csum_features = BGE_CSUM_FEATURES;
 +	if (sc->bge_forced_udpcsum != 0)
 +		sc->bge_csum_features |= CSUM_UDP;
 +
  	/* Set up ifnet structure */
  	ifp = sc->bge_ifp = if_alloc(IFT_ETHER);
  	if (ifp == NULL) {
 @@ -2818,7 +2825,7 @@ bge_attach(device_t dev)
  	ifp->if_snd.ifq_drv_maxlen = BGE_TX_RING_CNT - 1;
  	IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen);
  	IFQ_SET_READY(&ifp->if_snd);
 -	ifp->if_hwassist = BGE_CSUM_FEATURES;
 +	ifp->if_hwassist = sc->bge_csum_features;
  	ifp->if_capabilities = IFCAP_HWCSUM | IFCAP_VLAN_HWTAGGING |
  	    IFCAP_VLAN_MTU;
  	if ((sc->bge_flags & BGE_FLAG_TSO) != 0) {
 @@ -2975,8 +2982,6 @@ again:
  		device_printf(sc->bge_dev, "couldn't set up irq\n");
  	}
  
 -	bge_add_sysctls(sc);
 -
  	return (0);
  
  fail:
 @@ -3956,7 +3961,7 @@ bge_encap(struct bge_softc *sc, struct m
  			return (ENOBUFS);
  		csum_flags |= BGE_TXBDFLAG_CPU_PRE_DMA |
  		    BGE_TXBDFLAG_CPU_POST_DMA;
 -	} else if ((m->m_pkthdr.csum_flags & BGE_CSUM_FEATURES) != 0) {
 +	} else if ((m->m_pkthdr.csum_flags & sc->bge_csum_features) != 0) {
  		if (m->m_pkthdr.csum_flags & CSUM_IP)
  			csum_flags |= BGE_TXBDFLAG_IP_CSUM;
  		if (m->m_pkthdr.csum_flags & (CSUM_TCP | CSUM_UDP)) {
 @@ -4233,6 +4238,17 @@ bge_init_locked(struct bge_softc *sc)
  	/* Program VLAN tag stripping. */
  	bge_setvlan(sc);
  
 +	/* Override UDP checksum offloading. */
 +	if (sc->bge_forced_udpcsum == 0)
 +		sc->bge_csum_features &= ~CSUM_UDP;
 +	else
 +		sc->bge_csum_features |= CSUM_UDP;
 +	if (ifp->if_capabilities & IFCAP_TXCSUM &&
 +	    ifp->if_capenable & IFCAP_TXCSUM) {
 +		ifp->if_hwassist &= ~(BGE_CSUM_FEATURES | CSUM_UDP);
 +		ifp->if_hwassist |= sc->bge_csum_features;
 +	}
 +
  	/* Init RX ring. */
  	if (bge_init_rx_ring_std(sc) != 0) {
  		device_printf(sc->bge_dev, "no memory for std Rx buffers.\n");
 @@ -4558,9 +4574,9 @@ bge_ioctl(struct ifnet *ifp, u_long comm
  			ifp->if_capenable ^= IFCAP_HWCSUM;
  			if (IFCAP_HWCSUM & ifp->if_capenable &&
  			    IFCAP_HWCSUM & ifp->if_capabilities)
 -				ifp->if_hwassist |= BGE_CSUM_FEATURES;
 +				ifp->if_hwassist |= sc->bge_csum_features;
  			else
 -				ifp->if_hwassist &= ~BGE_CSUM_FEATURES;
 +				ifp->if_hwassist &= ~sc->bge_csum_features;
  		}
  
  		if ((mask & IFCAP_TSO4) != 0 &&
 @@ -4936,6 +4952,24 @@ bge_add_sysctls(struct bge_softc *sc)
  	    "Number of fragmented TX buffers of a frame allowed before "
  	    "forced collapsing");
  
 +	/*
 +	 * It seems all Broadcom controllers have a bug that can generate UDP
 +	 * datagrams with checksum value 0 when TX UDP checksum offloading is
 +	 * enabled.  Generating UDP checksum value 0 is RFC 768 violation.
 +	 * Even though the probability of generating such UDP datagrams is
 +	 * low, I don't want to see FreeBSD boxes to inject such datagrams
 +	 * into network so disable UDP checksum offloading by default.  Users
 +	 * still override this behavior by setting a sysctl variable,
 +	 * dev.bge.0.forced_udpcsum.
 +	 */
 +	sc->bge_forced_udpcsum = 0;
 +	snprintf(tn, sizeof(tn), "dev.bge.%d.bge_forced_udpcsum", unit);
 +	TUNABLE_INT_FETCH(tn, &sc->bge_forced_udpcsum);
 +	SYSCTL_ADD_INT(ctx, children, OID_AUTO, "forced_udpcsum",
 +	    CTLFLAG_RW, &sc->bge_forced_udpcsum, 0,
 +	    "Enable UDP checksum offloading even if controller can "
 +	    "generate UDP checksum value 0");
 +
  	if (BGE_IS_5705_PLUS(sc))
  		return;
  
 
 Modified: stable/7/sys/dev/bge/if_bgereg.h
 ==============================================================================
 --- stable/7/sys/dev/bge/if_bgereg.h	Fri Sep 24 17:59:58 2010	(r213109)
 +++ stable/7/sys/dev/bge/if_bgereg.h	Fri Sep 24 18:01:43 2010	(r213110)
 @@ -2644,6 +2644,8 @@ struct bge_softc {
  	int			bge_link_evt;	/* pending link event */
  	int			bge_timer;
  	int			bge_forced_collapse;
 +	int			bge_forced_udpcsum;
 +	int			bge_csum_features;
  	struct callout		bge_stat_ch;
  	uint32_t		bge_rx_discards;
  	uint32_t		bge_tx_discards;
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: yongari 
State-Changed-When: Fri Sep 24 19:49:15 UTC 2010 
State-Changed-Why:  
MFC to stable/8 and stable/7 done. 
Thanks for reporting! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=104826 
>Unformatted:
