From nobody@FreeBSD.org  Fri May  3 13:38:05 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115])
	by hub.freebsd.org (Postfix) with ESMTP id 191F63BB
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  3 May 2013 13:38:05 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [69.147.83.34])
	by mx1.freebsd.org (Postfix) with ESMTP id 0C05610FA
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  3 May 2013 13:38:05 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.5/8.14.5) with ESMTP id r43Dc4c2026497
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 3 May 2013 13:38:04 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.5/8.14.5/Submit) id r43Dc4RZ026496;
	Fri, 3 May 2013 13:38:04 GMT
	(envelope-from nobody)
Message-Id: <201305031338.r43Dc4RZ026496@red.freebsd.org>
Date: Fri, 3 May 2013 13:38:04 GMT
From: Luiz Otavio O Souza <loos.br@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [patch] [arge] arge_stop() doesn't clean the tx ring properly
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         178319
>Category:       kern
>Synopsis:       [patch] [arge] arge_stop() doesn't clean the tx ring properly
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    loos
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri May 03 13:40:00 UTC 2013
>Closed-Date:    Sun Sep 08 14:33:31 UTC 2013
>Last-Modified:  Sun Sep 08 14:33:31 UTC 2013
>Originator:     Luiz Otavio O Souza
>Release:        -head r250121
>Organization:
>Environment:
FreeBSD rb433 10.0-CURRENT FreeBSD 10.0-CURRENT #61 r250121M: Fri May  3 09:45:51 BRT 2013     root@devel:/data/rb/rb433/obj/mips.mips/data/rb/rb433/src/sys/RSPRO  mips
>Description:
While debugging the bootp race with if_arge (kern/178318) i have observed another weird thing, i see some packets being queued on the tx ring and when the TX_PKT_SENT interrupt arrives the tx ring was reseted. After a while the driver panics with what looks like a leakage.

After some debug i found that if arge_stop() wont properly clean the tx ring.

So you may end with some leak of dma tags and mbufs on the following case (which was triggered by the bootp problem):

- arge_start() -> queue packets on tx ring;
- arge_init() ->disable interrupts -> arge_stop() -> wont reset properly the tx ring -> enable interrupts;
- TX_PKT_SENT interrupt received -> arge_tx() wont see anything to clear.

If this happens for a while the driver will panic due to the leakage of dma tags.
>How-To-Repeat:
Add some printf() debug to arge_init_locked():


Index: mips/atheros/if_arge.c
===================================================================
--- mips/atheros/if_arge.c      (revision 250121)
+++ mips/atheros/if_arge.c      (working copy)
@@ -1006,6 +1006,7 @@
 
        ARGE_LOCK_ASSERT(sc);
 
+printf("%s: called\n", __func__);
        arge_stop(sc);
 
        /* Init circular RX list. */



Configure the RSPRO to boot from bootp/nfs:


Index: sys/mips/conf/RSPRO
===================================================================
--- sys/mips/conf/RSPRO (revision 250121)
+++ sys/mips/conf/RSPRO (working copy)
@@ -28,3 +28,12 @@
 # Boot off of flash
 options                ROOTDEVNAME=\"ufs:redboot/rootfs.uzip\"
 
+options                NFSCL
+options                NFS_ROOT
+options                BOOTP
+options                BOOTP_NFSROOT
+options                BOOTP_NFSV3
+options                BOOTP_WIRED_TO=arge0
+options                BOOTP_COMPAT
+
+

>Fix:
Make arge_stop() properly clean the tx ring.

Patch attached with submission follows:

Index: sys/mips/atheros/if_arge.c
===================================================================
--- sys/mips/atheros/if_arge.c	(revision 250121)
+++ sys/mips/atheros/if_arge.c	(working copy)
@@ -142,6 +142,7 @@
 static int arge_rx_ring_init(struct arge_softc *);
 static void arge_rx_ring_free(struct arge_softc *sc);
 static int arge_tx_ring_init(struct arge_softc *);
+static void arge_tx_ring_free(struct arge_softc *);
 #ifdef DEVICE_POLLING
 static int arge_poll(struct ifnet *, enum poll_cmd, int);
 #endif
@@ -1264,6 +1265,7 @@
 	/* Flush FIFO and free any existing mbufs */
 	arge_flush_ddr(sc);
 	arge_rx_ring_free(sc);
+	arge_tx_ring_free(sc);
 }
 
 
@@ -1694,6 +1696,27 @@
 }
 
 /*
+ * Free the Tx ring, unload any pending dma transaction and eat the mbuf.
+ */
+static void
+arge_tx_ring_free(struct arge_softc *sc)
+{
+	struct arge_txdesc	*txd;
+	int			i;
+
+	/* Free the Tx buffers. */
+	for (i = 0; i < ARGE_TX_RING_COUNT; i++) {
+		txd = &sc->arge_cdata.arge_txdesc[i];   
+		if (txd->tx_dmamap)
+			bus_dmamap_unload(sc->arge_cdata.arge_tx_tag,
+			    txd->tx_dmamap);
+		if (txd->tx_m)
+			m_freem(txd->tx_m);
+		txd->tx_m = NULL;
+	}
+}
+
+/*
  * Initialize the RX descriptors and allocate mbufs for them. Note that
  * we arrange the descriptors in a closed ring, so that the last descriptor
  * points back to the first.


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-mips 
Responsible-Changed-By: adrian 
Responsible-Changed-When: Thu May 16 03:23:25 UTC 2013 
Responsible-Changed-Why:  
assign to owner. 


http://www.freebsd.org/cgi/query-pr.cgi?pr=178319 
Responsible-Changed-From-To: freebsd-mips->loos 
Responsible-Changed-By: loos 
Responsible-Changed-When: Wed Jul 31 13:47:02 UTC 2013 
Responsible-Changed-Why:  
Take it. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=178319 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/178319: commit references a PR
Date: Fri,  6 Sep 2013 12:47:21 +0000 (UTC)

 Author: loos
 Date: Fri Sep  6 12:47:14 2013
 New Revision: 255300
 URL: http://svnweb.freebsd.org/changeset/base/255300
 
 Log:
   Fix the leakage of dma tags on if_arge.  The leak occur when arge_start()
   add some packet(s) to tx ring and arge_stop() is called before receive the
   sent packet interrupt from hardware.  Fix arge_stop() to unload the in use
   dma tags and free the associated mbuf.
   
   PR:		178319, 163670
   Approved by:	adrian (mentor)
 
 Modified:
   head/sys/mips/atheros/if_arge.c
 
 Modified: head/sys/mips/atheros/if_arge.c
 ==============================================================================
 --- head/sys/mips/atheros/if_arge.c	Fri Sep  6 12:45:08 2013	(r255299)
 +++ head/sys/mips/atheros/if_arge.c	Fri Sep  6 12:47:14 2013	(r255300)
 @@ -142,6 +142,7 @@ static int arge_resume(device_t);
  static int arge_rx_ring_init(struct arge_softc *);
  static void arge_rx_ring_free(struct arge_softc *sc);
  static int arge_tx_ring_init(struct arge_softc *);
 +static void arge_tx_ring_free(struct arge_softc *);
  #ifdef DEVICE_POLLING
  static int arge_poll(struct ifnet *, enum poll_cmd, int);
  #endif
 @@ -1278,6 +1279,7 @@ arge_stop(struct arge_softc *sc)
  	/* Flush FIFO and free any existing mbufs */
  	arge_flush_ddr(sc);
  	arge_rx_ring_free(sc);
 +	arge_tx_ring_free(sc);
  }
  
  
 @@ -1708,6 +1710,30 @@ arge_tx_ring_init(struct arge_softc *sc)
  }
  
  /*
 + * Free the Tx ring, unload any pending dma transaction and free the mbuf.
 + */
 +static void
 +arge_tx_ring_free(struct arge_softc *sc)
 +{
 +	struct arge_txdesc	*txd;
 +	int			i;
 +
 +	/* Free the Tx buffers. */
 +	for (i = 0; i < ARGE_TX_RING_COUNT; i++) {
 +		txd = &sc->arge_cdata.arge_txdesc[i];
 +		if (txd->tx_dmamap) {
 +			bus_dmamap_sync(sc->arge_cdata.arge_tx_tag,
 +			    txd->tx_dmamap, BUS_DMASYNC_POSTWRITE);
 +			bus_dmamap_unload(sc->arge_cdata.arge_tx_tag,
 +			    txd->tx_dmamap);
 +		}
 +		if (txd->tx_m)
 +			m_freem(txd->tx_m);
 +		txd->tx_m = NULL;
 +	}
 +}
 +
 +/*
   * Initialize the RX descriptors and allocate mbufs for them. Note that
   * we arrange the descriptors in a closed ring, so that the last descriptor
   * points back to the first.
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: loos 
State-Changed-When: Sun Sep 8 14:33:31 UTC 2013 
State-Changed-Why:  
Committed a slightly better version of the patch. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=178319 
>Unformatted:
