From nobody@FreeBSD.org  Mon Apr  9 15:58:29 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id B642616A409
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  9 Apr 2007 15:58:29 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.freebsd.org (Postfix) with ESMTP id A9E3613C4BC
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  9 Apr 2007 15:58:29 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id l39FwTVA059643
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 9 Apr 2007 15:58:29 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id l39FrRc2058443;
	Mon, 9 Apr 2007 15:53:27 GMT
	(envelope-from nobody)
Message-Id: <200704091553.l39FrRc2058443@www.freebsd.org>
Date: Mon, 9 Apr 2007 15:53:27 GMT
From: Vladimir Ivanov<wawa@yandex-team.ru>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Serious bug in bce (and bfe?) ethernet driver
X-Send-Pr-Version: www-3.0

>Number:         111415
>Category:       kern
>Synopsis:       [bce] [patch] Serious bug in bce (and bfe?) ethernet driver
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    davidch
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Apr 09 16:00:08 GMT 2007
>Closed-Date:    Mon Jan 14 21:18:07 UTC 2008
>Last-Modified:  Mon Jan 14 21:18:07 UTC 2008
>Originator:     Vladimir Ivanov
>Release:        RELENG_6
>Organization:
Yandex LLC
>Environment:
FreeBSD bernoulli.yandex.net 6.2-STABLE FreeBSD 6.2-STABLE #26: Sat Apr  7 14:17:48 MSD 2007     root@bernoulli.yandex.net:/usr/obj/usr/src/sys/CORE-RTR-RELENG_6  i386

>Description:
We have reported serious bug with em driver
(http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/87418) one year and half ago.

The bug cause RX queue corruption and interface stuck.

Unfortunately some freebsd ethernet drivers reproduced this bug.
You can see same kind of bug in bce and ixgb and bfe(?). 

I've attached draft patch for bce driver (RELENG_6) to solve the problem.

We've started to test this patch revision one week before on twenty boxes.
They seems to be Ok still.


>How-To-Repeat:

>Fix:


Patch attached with submission follows:

--- if_bce.c.orig	Fri Mar 16 00:58:25 2007
+++ if_bce.c	Thu Apr  5 02:57:39 2007
@@ -3943,6 +3943,8 @@ bce_rx_intr(struct bce_softc *sc)
 		unsigned int len;
 		u32 status;
 
+		m = NULL;
+
 		/* Convert the producer/consumer indices to an actual rx_bd index. */
 		sw_chain_cons = RX_CHAIN_IDX(sw_cons);
 		sw_chain_prod = RX_CHAIN_IDX(sw_prod);
@@ -4131,16 +4133,30 @@ bce_rx_intr(struct bce_softc *sc)
 			ifp->if_ipackets++;
 			DBPRINT(sc, BCE_VERBOSE_RECV, "%s(): Passing received frame up.\n",
 				__FUNCTION__);
-			BCE_UNLOCK(sc);
-			(*ifp->if_input)(ifp, m);
-			DBRUNIF(1, sc->rx_mbuf_alloc--);
-			BCE_LOCK(sc);
 
 bce_rx_int_next_rx:
 			sw_prod = NEXT_RX_BD(sw_prod);
 		}
 
 		sw_cons = NEXT_RX_BD(sw_cons);
+
+		if (m) {
+			sc->rx_cons = sw_cons;
+			sc->rx_prod = sw_prod;
+			sc->rx_prod_bseq = sw_prod_bseq;
+
+			BCE_UNLOCK(sc);
+			(*ifp->if_input)(ifp, m);
+			BCE_LOCK(sc);
+                        DBRUNIF(1, sc->rx_mbuf_alloc--);
+
+		        sw_cons = sc->rx_cons;
+      			sw_prod = sc->rx_prod;
+		        sw_prod_bseq = sc->rx_prod_bseq;
+			hw_cons = sc->hw_rx_cons = sblk->status_rx_quick_consumer_index0;
+			if ((hw_cons & USABLE_RX_BD_PER_PAGE) == USABLE_RX_BD_PER_PAGE)
+				hw_cons++;
+		}
 
 		/* Refresh hw_cons to see if there's new work */
 		if (sw_cons == hw_cons) {

>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: delphij 
State-Changed-When: Mon Jan 14 21:16:39 UTC 2008 
State-Changed-Why:  
David has committed the proposed patch as rev 1.30 and MFC'ed 
to RELENG_6 as rev 1.2.2.15. 


Responsible-Changed-From-To: freebsd-bugs->davidch 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Mon Jan 14 21:16:39 UTC 2008 
Responsible-Changed-Why:  
Over to davidch@ 

http://www.freebsd.org/cgi/query-pr.cgi?pr=111415 
>Unformatted:
