From nox@jelal.kn-bremen.de  Wed Jan 12 20:16:05 2011
Return-Path: <nox@jelal.kn-bremen.de>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4B72F1065674;
	Wed, 12 Jan 2011 20:16:05 +0000 (UTC)
	(envelope-from nox@jelal.kn-bremen.de)
Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116])
	by mx1.freebsd.org (Postfix) with ESMTP id 9B0338FC17;
	Wed, 12 Jan 2011 20:16:03 +0000 (UTC)
Received: by smtp.kn-bremen.de (Postfix, from userid 10)
	id 5B0CA1E000C6; Wed, 12 Jan 2011 20:58:11 +0100 (CET)
Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1])
	by triton8.kn-bremen.de (8.14.4/8.14.3) with ESMTP id p0CJu024078590;
	Wed, 12 Jan 2011 20:56:00 +0100 (CET)
	(envelope-from nox@triton8.kn-bremen.de)
Received: (from nox@localhost)
	by triton8.kn-bremen.de (8.14.4/8.14.3/Submit) id p0CJtx5B078589;
	Wed, 12 Jan 2011 20:55:59 +0100 (CET)
	(envelope-from nox)
Message-Id: <201101121955.p0CJtx5B078589@triton8.kn-bremen.de>
Date: Wed, 12 Jan 2011 20:55:59 +0100 (CET)
From: Juergen Lock <nox@jelal.kn-bremen.de>
Reply-To: Juergen Lock <nox@jelal.kn-bremen.de>
To: FreeBSD-gnats-submit@freebsd.org
Cc: moonlightakkiy@yahoo.ca, freebsd-net@freebsd.org
Subject: [run] [panic] [patch] Workaround for use-after-free panic
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         153938
>Category:       kern
>Synopsis:       [run] [panic] [patch] Workaround for use-after-free panic
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    hselasky
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jan 12 20:20:04 UTC 2011
>Closed-Date:    Sun Jul 24 22:17:29 UTC 2011
>Last-Modified:  Sun Jul 24 22:17:29 UTC 2011
>Originator:     Juergen Lock
>Release:        FreeBSD 8.1-RC2 amd64
>Organization:
me?  organized??
>Environment:
System: FreeBSD triton8.kn-bremen.de 8.1-RC2 FreeBSD 8.1-RC2 #9: Wed Sep 1 21:53:36 CEST 2010 nox@triton8.kn-bremen.de:/usr/obj/data2v/home/nox/src-r81/src/sys/TRITON8U amd64

	Yes this is an older stable/8 checkout but if_run(4) is
	checked out from head.

>Description:
	Running the nic in hostap mode with wpa2 I once every few
	weeks got the following crash:

#0  doadump () at pcpu.h:223
223	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:223
#1  0xffffffff805f0719 in boot (howto=260)
    at /data2v/home/nox/src-r81/src/sys/kern/kern_shutdown.c:416
#2  0xffffffff805f0b6c in panic (fmt=Variable "fmt" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/kern/kern_shutdown.c:590
#3  0xffffffff808e4e0d in trap_fatal (frame=0xc, eva=Variable "eva" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/amd64/amd64/trap.c:777
#4  0xffffffff808e51f4 in trap_pfault (frame=0xffffff80ec121aa0, usermode=0)
    at /data2v/home/nox/src-r81/src/sys/amd64/amd64/trap.c:693
#5  0xffffffff808e5a7e in trap (frame=0xffffff80ec121aa0)
    at /data2v/home/nox/src-r81/src/sys/amd64/amd64/trap.c:451
#6  0xffffffff808ca953 in calltrap ()
    at /data2v/home/nox/src-r81/src/sys/amd64/amd64/exception.S:223
#7  0xffffffff81072ac6 in run_drain_fifo (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2245
#8  0xffffffff81072bc3 in run_ratectl_cb (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2210
#9  0xffffffff8062e543 in taskqueue_run (queue=0xffffff0005f42380)
    at /data2v/home/nox/src-r81/src/sys/kern/subr_taskqueue.c:239
#10 0xffffffff8062e7c6 in taskqueue_thread_loop (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/kern/subr_taskqueue.c:360
---Type <return> to continue, or q <return> to quit---
#11 0xffffffff805c64a8 in fork_exit (
    callout=0xffffffff8062e780 <taskqueue_thread_loop>, 
    arg=0xffffff8000b130b8, frame=0xffffff80ec121c80)
    at /data2v/home/nox/src-r81/src/sys/kern/kern_fork.c:844
#12 0xffffffff808cae2e in fork_trampoline ()
    at /data2v/home/nox/src-r81/src/sys/amd64/amd64/exception.S:562
#13 0x0000000000000000 in ?? ()
#14 0x0000000000000000 in ?? ()
#15 0x0000000000000000 in ?? ()
#16 0x0000000000000000 in ?? ()
#17 0x0000000000000000 in ?? ()
#18 0x0000000000000000 in ?? ()
#19 0x0000000000000000 in ?? ()
#20 0x0000000000000000 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x0000000000000000 in ?? ()
#23 0x0000000000000000 in ?? ()
#24 0x0000000000000000 in ?? ()
#25 0x0000000000000000 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x0000000000000000 in ?? ()
---Type <return> to continue, or q <return> to quit---
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000f37000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0xffffff00078c47c0 in ?? ()
#40 0xffffffff80cac9c0 in affinity ()
#41 0xffffff00018837c0 in ?? ()
#42 0xffffff80ec121710 in ?? ()
#43 0xffffff80ec1216c8 in ?? ()
#44 0xffffff00078c47c0 in ?? ()
#45 0xffffffff8061471a in sched_switch (td=0xffffff8000b130b8, 
    newtd=0xffffffff8062e780, flags=Variable "flags" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1844
Previous frame inner to this frame (corrupt stack?)
(kgdb) fr 7
#7  0xffffffff81072ac6 in run_drain_fifo (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2245
2245			ni = sc->sc_ni[wcid];
(kgdb) p wcid
$1 = 1 '\001'
(kgdb) p sc->sc_ni
$2 = {0x0, 0xffffff8001676000, 0x0 <repeats 63 times>}
(kgdb) p sc->sc_ni[1]
$3 = (struct ieee80211_node *) 0xffffff8001676000
(kgdb) p *sc->sc_ni[1]
Cannot access memory at address 0xffffff8001676000
(kgdb) up
#8  0xffffffff81072bc3 in run_ratectl_cb (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2210
2210			run_drain_fifo(sc);
(kgdb) p sc->sc_ni
$4 = {0x0, 0xffffff8001676000, 0x0 <repeats 63 times>}
(kgdb) l run_drain_fifo
2216			usb_callout_reset(&sc->ratectl_ch, hz, run_ratectl_to, sc);
2217	}
2218	
2219	static void
2220	run_drain_fifo(void *arg)
2221	{
2222		struct run_softc *sc = arg;
2223		struct ifnet *ifp = sc->sc_ifp;
2224		struct ieee80211_node *ni = sc->sc_ni[0];	/* make compiler happy */
2225		uint32_t stat;
(kgdb) l
2226		int retrycnt = 0;
2227		uint8_t wcid, mcs, pid;
2228	
2229		RUN_LOCK_ASSERT(sc, MA_OWNED);
2230	
2231		for (;;) {
2232			/* drain Tx status FIFO (maxsize = 16) */
2233			run_read(sc, RT2860_TX_STAT_FIFO, &stat);
2234			DPRINTFN(4, "tx stat 0x%08x\n", stat);
2235			if (!(stat & RT2860_TXQ_VLD))
(kgdb) 
2236				break;
2237	
2238			wcid = (stat >> RT2860_TXQ_WCID_SHIFT) & 0xff;
2239	
2240			/* if no ACK was requested, no feedback is available */
2241			if (!(stat & RT2860_TXQ_ACKREQ) || wcid > RT2870_WCID_MAX ||
2242			    wcid == 0)
2243				continue;
2244	
2245			ni = sc->sc_ni[wcid];
(kgdb) 
2246			if (ni->ni_rctls == NULL)
2247				continue;
2248	
2249			/* update per-STA AMRR stats */
2250			if (stat & RT2860_TXQ_OK) {
2251				/*
2252				 * Check if there were retries, ie if the Tx
2253				 * success rate is different from the requested
2254				 * rate. Note that it works only because we do
2255				 * not allow rate fallback from OFDM to CCK.
(kgdb) 
2256				 */
2257				mcs = (stat >> RT2860_TXQ_MCS_SHIFT) & 0x7f;
2258				pid = (stat >> RT2860_TXQ_PID_SHIFT) & 0xf;
2259				if (mcs + 1 != pid)
2260					retrycnt = 1;
2261				ieee80211_ratectl_tx_complete(ni->ni_vap, ni,
2262				    IEEE80211_RATECTL_TX_SUCCESS,
2263				    &retrycnt, NULL);
2264			} else {
2265				retrycnt = 1;
(kgdb) 
2266				ieee80211_ratectl_tx_complete(ni->ni_vap, ni,
2267				    IEEE80211_RATECTL_TX_FAILURE,
2268				    &retrycnt, NULL);
2269				ifp->if_oerrors++;
2270			}
2271		}
2272		DPRINTFN(3, "count=%d\n", sc->fifo_cnt);
2273	
2274		sc->fifo_cnt = 0;
2275	}
(kgdb) up
#9  0xffffffff8062e543 in taskqueue_run (queue=0xffffff0005f42380)
    at /data2v/home/nox/src-r81/src/sys/kern/subr_taskqueue.c:239
239			task->ta_func(task->ta_context, pending);
(kgdb) p task
$5 = (struct task *) 0xffffff8000a8be38
(kgdb) p *task
$6 = {ta_link = {stqe_next = 0x0}, ta_pending = 0, ta_priority = 0, 
  ta_func = 0xffffffff81072b60 <run_ratectl_cb>, 
  ta_context = 0xffffff8000a89000}
(kgdb) l run_ratectl_cb
2184	}
2185	
2186	/* ARGSUSED */
2187	static void
2188	run_ratectl_cb(void *arg, int pending)
2189	{
2190		struct run_softc *sc = arg;
2191		struct ieee80211com *ic = sc->sc_ifp->if_l2com;
2192		struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps);
2193	
(kgdb) down
#8  0xffffffff81072bc3 in run_ratectl_cb (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2210
2210			run_drain_fifo(sc);
(kgdb) l run_ratectl_cb
2184	}
2185	
2186	/* ARGSUSED */
2187	static void
2188	run_ratectl_cb(void *arg, int pending)
2189	{
2190		struct run_softc *sc = arg;
2191		struct ieee80211com *ic = sc->sc_ifp->if_l2com;
2192		struct ieee80211vap *vap = TAILQ_FIRST(&ic->ic_vaps);
2193	
(kgdb) l
2194		if (vap == NULL)
2195			return;
2196	
2197		if (sc->rvp_cnt <= 1 && vap->iv_opmode == IEEE80211_M_STA)
2198			run_iter_func(sc, vap->iv_bss);
2199		else {
2200			/*
2201			 * run_reset_livelock() doesn't do anything with AMRR,
2202			 * but Ralink wants us to call it every 1 sec. So, we
2203			 * piggyback here rather than creating another callout.
(kgdb) p sc->rvp_cnt
$7 = 1 '\001'
(kgdb) l
2204			 * Livelock may occur only in HOSTAP or IBSS mode
2205			 * (when h/w is sending beacons).
2206			 */
2207			RUN_LOCK(sc);
2208			run_reset_livelock(sc);
2209			/* just in case, there are some stats to drain */
2210			run_drain_fifo(sc);
2211			RUN_UNLOCK(sc);
2212			ieee80211_iterate_nodes(&ic->ic_sta, run_iter_func, sc);
2213		}
(kgdb) down
#7  0xffffffff81072ac6 in run_drain_fifo (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2245
2245			ni = sc->sc_ni[wcid];
(kgdb) up
#8  0xffffffff81072bc3 in run_ratectl_cb (arg=Variable "arg" is not available.
)
    at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2210
2210			run_drain_fifo(sc);
(kgdb) l
2205			 * (when h/w is sending beacons).
2206			 */
2207			RUN_LOCK(sc);
2208			run_reset_livelock(sc);
2209			/* just in case, there are some stats to drain */
2210			run_drain_fifo(sc);
2211			RUN_UNLOCK(sc);
2212			ieee80211_iterate_nodes(&ic->ic_sta, run_iter_func, sc);
2213		}
2214	
(kgdb) l
2215		if(sc->ratectl_run != RUN_RATECTL_OFF)
2216			usb_callout_reset(&sc->ratectl_ch, hz, run_ratectl_to, sc);
2217	}
2218	
2219	static void
2220	run_drain_fifo(void *arg)
2221	{
2222		struct run_softc *sc = arg;
2223		struct ifnet *ifp = sc->sc_ifp;
2224		struct ieee80211_node *ni = sc->sc_ni[0];	/* make compiler happy */
(kgdb) 
2225		uint32_t stat;
2226		int retrycnt = 0;
2227		uint8_t wcid, mcs, pid;
2228	
2229		RUN_LOCK_ASSERT(sc, MA_OWNED);
2230	
2231		for (;;) {
2232			/* drain Tx status FIFO (maxsize = 16) */
2233			run_read(sc, RT2860_TX_STAT_FIFO, &stat);
2234			DPRINTFN(4, "tx stat 0x%08x\n", stat);
(kgdb) p sc->fifo_cnt
$8 = 1 '\001'
(kgdb) l
2235			if (!(stat & RT2860_TXQ_VLD))
2236				break;
2237	
2238			wcid = (stat >> RT2860_TXQ_WCID_SHIFT) & 0xff;
2239	
2240			/* if no ACK was requested, no feedback is available */
2241			if (!(stat & RT2860_TXQ_ACKREQ) || wcid > RT2870_WCID_MAX ||
2242			    wcid == 0)
2243				continue;
2244	
(kgdb) l
2245			ni = sc->sc_ni[wcid];
2246			if (ni->ni_rctls == NULL)
2247				continue;
2248	
2249			/* update per-STA AMRR stats */
2250			if (stat & RT2860_TXQ_OK) {
2251				/*
2252				 * Check if there were retries, ie if the Tx
2253				 * success rate is different from the requested
2254				 * rate. Note that it works only because we do
(kgdb) p vap->iv_opmode
Variable "vap" is not available.
(kgdb) p ic->ic_vaps
$9 = {tqh_first = 0xffffff0007800000, tqh_last = 0xffffff0007800048}
(kgdb) p ic->ic_vaps->tqh_first
$10 = (struct ieee80211vap *) 0xffffff0007800000
(kgdb) p ic->ic_vaps->tqh_first->iv_opmode
$11 = IEEE80211_M_HOSTAP
(kgdb) p ic->ic_vaps->tqh_last->iv_opmode
Cannot access memory at address 0x2f0
(kgdb) p ic->ic_vaps->tqh_last
$12 = (struct ieee80211vap **) 0xffffff0007800048
(kgdb) p *ic->ic_vaps->tqh_last
$13 = (struct ieee80211vap *) 0x0
(kgdb) q

Script done on Tue Jan  4 09:23:48 2011
>How-To-Repeat:
	Setup if_run(4) in hostap mode, wait a few weeks...
	(I only have one smartphone using the wifi, maybe if
	you have a bigger network it'll happen more often?)

>Fix:

	I don't really know the wifi code so the following patch
	is likely not the `proper' fix (and it also still has
	diagnostic code that shouldn't be committed as is), but at
	least it fixed the panic for me, I just finally got the

		run0: drain_fifo ni=NULL wcid=1

	message I added for the condition that previously caused
	the panic, and the nic kept working.  (The panic happened
	when sc->sc_ni[wcid] was accessed by run_drain_fifo() after
	it had been free'd, so I hooked into ic->ic_node_cleanup
	to set it to NULL before it gets free'd and added a check
	for NULL with the above message to run_drain_fifo().)

Index: src/sys/dev/usb/wlan/if_run.c
===================================================================
RCS file: /home/scvs/src/sys/dev/usb/wlan/if_run.c,v
retrieving revision 1.17
diff -u -p -r1.17 if_run.c
--- src/sys/dev/usb/wlan/if_run.c	6 Nov 2010 18:17:20 -0000	1.17
+++ src/sys/dev/usb/wlan/if_run.c	7 Jan 2011 00:58:35 -0000
@@ -341,6 +341,7 @@ static const char *run_get_rf(int);
 static int	run_read_eeprom(struct run_softc *);
 static struct ieee80211_node *run_node_alloc(struct ieee80211vap *,
 			    const uint8_t mac[IEEE80211_ADDR_LEN]);
+static void	run_node_cleanup(struct ieee80211_node *ni);
 static int	run_media_change(struct ifnet *);
 static int	run_newstate(struct ieee80211vap *, enum ieee80211_state, int);
 static int	run_wme_update(struct ieee80211com *);
@@ -673,6 +674,8 @@ run_attach(device_t self)
 	ic->ic_scan_end = run_scan_end;
 	ic->ic_set_channel = run_set_channel;
 	ic->ic_node_alloc = run_node_alloc;
+	sc->sc_node_cleanup = ic->ic_node_cleanup;
+	ic->ic_node_cleanup = run_node_cleanup;
 	ic->ic_newassoc = run_newassoc;
 	//ic->ic_updateslot = run_updateslot;
 	ic->ic_update_mcast = run_update_mcast;
@@ -2243,7 +2246,14 @@ run_drain_fifo(void *arg)
 			continue;
 
 		ni = sc->sc_ni[wcid];
-		if (ni->ni_rctls == NULL)
+#if 1
+		static struct ieee80211_node *lastni;
+		if (ni == NULL && lastni)
+			device_printf(sc->sc_dev, "drain_fifo ni=NULL wcid=%d\n",
+				wcid);
+		lastni = ni;
+#endif
+		if (ni == NULL || ni->ni_rctls == NULL)
 			continue;
 
 		/* update per-STA AMRR stats */
@@ -2373,10 +2383,12 @@ run_newassoc(struct ieee80211_node *ni, 
 		ieee80211_runtask(ic, &sc->cmdq_task);
 	}
 
-	DPRINTF("new assoc isnew=%d associd=%x addr=%s\n",
-	    isnew, ni->ni_associd, ether_sprintf(ni->ni_macaddr));
+	//DPRINTF("new assoc isnew=%d associd=%x addr=%s\n",
+	device_printf(sc->sc_dev, "new assoc isnew=%d associd=%x addr=%s ni=%p\n",
+	    isnew, ni->ni_associd, ether_sprintf(ni->ni_macaddr), ni);
 
 	ieee80211_ratectl_node_init(ni);
+	rn->wcid = wcid;
 	sc->sc_ni[wcid] = ni;
 
 	for (i = 0; i < rs->rs_nrates; i++) {
@@ -2412,6 +2424,39 @@ run_newassoc(struct ieee80211_node *ni, 
 	usb_callout_reset(&sc->ratectl_ch, hz, run_ratectl_to, sc);
 }
 
+static void
+run_node_cleanup(struct ieee80211_node *ni)
+{
+	struct run_node *rn = (void *)ni;
+	struct ieee80211vap *vap = ni->ni_vap;
+	struct ieee80211com *ic = vap->iv_ic;
+	struct run_softc *sc = ic->ic_ifp->if_softc;
+	uint8_t wcid = RUN_AID2WCID(ni->ni_associd);
+
+	if (wcid == 0)
+		wcid = rn->wcid;
+	if (wcid > RT2870_WCID_MAX) {
+		device_printf(sc->sc_dev, "wcid=%d out of range\n", wcid);
+		sc->sc_node_cleanup(ni);
+		return;
+	}
+
+	//DPRINTF("node_cleanup wcid=%d addr=%s\n",
+	device_printf(sc->sc_dev, "node_cleanup wcid=%d addr=%s ni=%p\n",
+	    wcid, ether_sprintf(vap->iv_opmode == IEEE80211_M_STA ?
+		    vap->iv_myaddr : ni->ni_macaddr), ni);
+
+	if (wcid > 0 && sc->sc_ni[wcid]) {
+		if (sc->sc_ni[wcid] != ni) {
+			device_printf(sc->sc_dev, "node_cleanup sc->sc_ni[wcid] %p != ni\n",
+				sc->sc_ni[wcid]);
+		} else {
+			sc->sc_ni[wcid] = NULL;
+		}
+	}
+	sc->sc_node_cleanup(ni);
+}
+
 /*
  * Return the Rx chain with the highest RSSI for a given frame.
  */
Index: src/sys/dev/usb/wlan/if_runvar.h
===================================================================
RCS file: /home/scvs/src/sys/dev/usb/wlan/if_runvar.h,v
retrieving revision 1.6
diff -u -p -r1.6 if_runvar.h
--- src/sys/dev/usb/wlan/if_runvar.h	14 Jun 2010 00:40:23 -0000	1.6
+++ src/sys/dev/usb/wlan/if_runvar.h	4 Jan 2011 08:48:13 -0000
@@ -106,6 +106,7 @@ struct run_node {
 	uint8_t			amrr_ridx;
 	uint8_t			mgt_ridx;
 	uint8_t			fix_ridx;
+	uint8_t			wcid;
 };
 
 struct run_cmdq {
@@ -164,6 +165,8 @@ struct run_softc {
 	int				(*sc_srom_read)(struct run_softc *,
 					    uint16_t, uint16_t *);
 
+	void 				(*sc_node_cleanup)(struct ieee80211_node *);
+
 	uint16_t			mac_ver;
 	uint16_t			mac_rev;
 	uint8_t				rf_rev;
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Jan 13 10:07:38 UTC 2011 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=153938 

From: PseudoCylon <moonlightakkiy@yahoo.ca>
To: bug-followup@freebsd.org, nox@jelal.kn-bremen.de
Cc:  
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free panic
Date: Thu, 13 Jan 2011 16:47:21 -0800 (PST)

 ----- Original Message ----
 > From: Juergen Lock <nox@jelal.kn-bremen.de>
 > To: FreeBSD-gnats-submit@freebsd.org
 > Cc: moonlightakkiy@yahoo.ca; freebsd-net@freebsd.org
 > Sent: Wed, January 12, 2011 12:55:59 PM
 > Subject: [run] [panic] [patch] Workaround for use-after-free panic
 > 
 > 
 > >Submitter-Id:     current-users
 > >Originator:    Juergen  Lock
 > >Organization:    me?   organized??
 > >Confidential:    no 
 > >Synopsis:    [run] [panic] [patch] Workaround for  use-after-free panic
 > >Severity:    
 > >Priority:    
 > >Category:     kern
 > >Class:         sw-bug
 > >Release:    FreeBSD 8.1-RC2  amd64
 > >Environment:
 > System: FreeBSD triton8.kn-bremen.de 8.1-RC2  FreeBSD 8.1-RC2 #9: Wed Sep 1 
 >21:53:36 CEST 2010 
 >nox@triton8.kn-bremen.de:/usr/obj/data2v/home/nox/src-r81/src/sys/TRITON8U  
 >amd64
 > 
 >     Yes this is an older stable/8 checkout but  if_run(4) is
 >     checked out from  head.
 > 
 > >Description:
 >     Running the nic in hostap  mode with wpa2 I once every few
 >     weeks got the following  crash:
 > 
 
 Hello,
 
 Thank you for the patch.
 
 I have applied it. Please try patched driver out.
 http://gitorious.org/run/run/trees/ratectl_fix/dev/usb/wlan
 
 I added locks to your patch, so saved pointers are more reliable.
 
 
 AK
 
 

From: Juergen Lock <nox@jelal.kn-bremen.de>
To: PseudoCylon <moonlightakkiy@yahoo.ca>
Cc: bug-followup@freebsd.org, nox@jelal.kn-bremen.de
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free
 panic
Date: Fri, 14 Jan 2011 18:36:50 +0100

 On Thu, Jan 13, 2011 at 04:47:21PM -0800, PseudoCylon wrote:
 > Hello,
 Hi!
 > 
 > Thank you for the patch.
 > 
  You're welcome! :)
 
 > I have applied it. Please try patched driver out.
 > http://gitorious.org/run/run/trees/ratectl_fix/dev/usb/wlan
 > 
 > I added locks to your patch, so saved pointers are more reliable.
 
  I see you removed the rn->wcid code, I guess I should have
 explained what it's for:  ni->ni_associd already gets zeroed before
 run_node_cleanup() is called so with your version no sc->sc_ni[wcid]
 ever gets set to NULL.
 
  New patch against git, this time with all the diagnostic code in #if 1:
 
 diff -upr run-run/dev/usb/wlan/if_run.c src/sys/dev/usb/wlan/if_run.c
 --- run-run/dev/usb/wlan/if_run.c	2011-01-14 00:35:23.000000000 +0100
 +++ src/sys/dev/usb/wlan/if_run.c	2011-01-14 17:17:50.000000000 +0100
 @@ -1694,12 +1694,12 @@ static void
  run_node_cleanup(struct ieee80211_node *ni)
  {
  	struct run_softc *sc = ni->ni_vap->iv_ic->ic_ifp->if_softc;
 +	struct run_node *rn = (void *)ni;
  	uint8_t wcid;
  
  	wcid = RUN_AID2WCID(ni->ni_associd);
  
 -	/* sc_ni[0] is not used */
 -	if ((wcid == 0) || (wcid > RT2870_WCID_MAX))
 +	if (wcid > RT2870_WCID_MAX)
  		goto done;
  
  	/*
 @@ -1707,13 +1707,27 @@ run_node_cleanup(struct ieee80211_node *
  	 * so lock conditionally
  	 */
  	if (mtx_owned(&sc->sc_mtx))
 -		sc->sc_ni[wcid] = NULL;
 +		if (wcid == 0)
 +			wcid = rn->wcid;
 +		/* sc_ni[0] is not used */
 +		if (wcid != 0 && wcid <= RT2870_WCID_MAX)
 +			sc->sc_ni[wcid] = NULL;
  	else {
  		RUN_LOCK(sc);
 -		sc->sc_ni[wcid] = NULL;
 +		if (wcid == 0)
 +			wcid = rn->wcid;
 +		/* sc_ni[0] is not used */
 +		if (wcid != 0 && wcid <= RT2870_WCID_MAX)
 +			sc->sc_ni[wcid] = NULL;
  		RUN_UNLOCK(sc);
  	}
  
 +#if 1
 +	device_printf(sc->sc_dev, "node_cleanup wcid=%d addr=%s ni=%p\n",
 +		wcid, ether_sprintf(ni->ni_vap->iv_opmode == IEEE80211_M_STA ?
 +			ni->ni_vap->iv_myaddr : ni->ni_macaddr), ni);
 +#endif
 +
  done:
  	sc->sc_node_cleanup(ni);
  }
 @@ -2279,6 +2293,13 @@ run_drain_fifo(void *arg)
  		    wcid == 0)
  			continue;
  
 +#if 1
 +		static struct ieee80211_node *lastni;
 +		if ((ni = sc->sc_ni[wcid]) == NULL && lastni)
 +			device_printf(sc->sc_dev, "drain_fifo ni=NULL wcid=%d\n",
 +				wcid);
 +		lastni = ni;
 +#endif
  		if ((sc->sc_ni[wcid] == NULL) ||
  		    (sc->sc_ni[wcid]->ni_rctls == NULL))
  			continue;
 @@ -2371,6 +2392,7 @@ run_newassoc_cb(void *arg)
  {
  	struct run_cmdq *cmdq = arg;
  	struct ieee80211_node *ni = cmdq->arg1;
 +	struct run_node *rn = (void *)ni;
  	struct run_softc *sc = ni->ni_vap->iv_ic->ic_ifp->if_softc;
  	uint8_t wcid = cmdq->wcid;
  
 @@ -2379,6 +2401,7 @@ run_newassoc_cb(void *arg)
  	run_write_region_1(sc, RT2860_WCID_ENTRY(wcid),
  	    ni->ni_macaddr, IEEE80211_ADDR_LEN);
  
 +	rn->wcid = wcid;
  	sc->sc_ni[wcid] = ni;
  }
  
 @@ -2416,8 +2439,13 @@ run_newassoc(struct ieee80211_node *ni, 
  		ieee80211_runtask(ic, &sc->cmdq_task);
  	}
  
 +#if 1
 +	device_printf(sc->sc_dev, "new assoc isnew=%d associd=%x addr=%s ni=%p\n",
 +	    isnew, ni->ni_associd, ether_sprintf(ni->ni_macaddr), ni);
 +#else
  	DPRINTF("new assoc isnew=%d associd=%x addr=%s\n",
  	    isnew, ni->ni_associd, ether_sprintf(ni->ni_macaddr));
 +#endif
  
  	ieee80211_ratectl_node_init(ni);
  
 diff -upr run-run/dev/usb/wlan/if_runvar.h src/sys/dev/usb/wlan/if_runvar.h
 --- run-run/dev/usb/wlan/if_runvar.h	2011-01-14 00:35:23.000000000 +0100
 +++ src/sys/dev/usb/wlan/if_runvar.h	2011-01-14 16:56:41.000000000 +0100
 @@ -106,6 +106,7 @@ struct run_node {
  	uint8_t			amrr_ridx;
  	uint8_t			mgt_ridx;
  	uint8_t			fix_ridx;
 +	uint8_t			wcid;
  };
  
  struct run_cmdq {

From: PseudoCylon <moonlightakkiy@yahoo.ca>
To: Juergen Lock <nox@jelal.kn-bremen.de>
Cc: bug-followup@freebsd.org, nox@jelal.kn-bremen.de
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free panic
Date: Sun, 16 Jan 2011 22:24:07 -0800 (PST)

 ----- Original Message ----
 > From: Juergen Lock <nox@jelal.kn-bremen.de>
 > To: PseudoCylon <moonlightakkiy@yahoo.ca>
 > Cc: bug-followup@freebsd.org; nox@jelal.kn-bremen.de
 > Sent: Fri, January 14, 2011 10:36:50 AM
 > Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free 
 >panic
 > 
 > On Thu, Jan 13, 2011 at 04:47:21PM -0800, PseudoCylon wrote:
 > >  Hello,
 > Hi!
 > > 
 > > Thank you for the patch.
 > > 
 >  You're  welcome! :)
 > 
 > > I have applied it. Please try patched driver  out.
 > > http://gitorious.org/run/run/trees/ratectl_fix/dev/usb/wlan
 > > 
 > > I added locks to your patch, so saved pointers are more  reliable.
 > 
 >  I see you removed the rn->wcid code, I guess I should  have
 > explained what it's for:  ni->ni_associd already gets zeroed  before
 > run_node_cleanup() is called so with your version no  sc->sc_ni[wcid]
 > ever gets set to NULL.
 > 
 
 You're right.
 
 > +        if (wcid ==  0)
 > +            wcid =  rn->wcid;
 
 
 Is there any reason to test "ni->ni_associd == 0"? We know it is 0.
 
 
 AK
 
 

From: Juergen Lock <nox@jelal.kn-bremen.de>
To: PseudoCylon <moonlightakkiy@yahoo.ca>
Cc: Juergen Lock <nox@jelal.kn-bremen.de>, bug-followup@freebsd.org
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free
 panic
Date: Mon, 17 Jan 2011 22:14:04 +0100

 On Sun, Jan 16, 2011 at 10:24:07PM -0800, PseudoCylon wrote:
 
 > > [...]
 > >  I see you removed the rn->wcid code, I guess I should  have
 > > explained what it's for:  ni->ni_associd already gets zeroed  before
 > > run_node_cleanup() is called so with your version no  sc->sc_ni[wcid]
 > > ever gets set to NULL.
 > > 
 > 
 > You're right.
 > 
 > > +        if (wcid ==  0)
 > > +            wcid =  rn->wcid;
 > 
 > 
 > Is there any reason to test "ni->ni_associd == 0"? We know it is 0.
 
 Oh I only left it in in case the surrounding code changes in the
 future, but I guess that's pretty unlikely.  So I agree the check
 can be removed...

From: PseudoCylon <moonlightakkiy@yahoo.ca>
To: bug-followup@freebsd.org, Juergen Lock <nox@jelal.kn-bremen.de>
Cc:  
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free panic
Date: Thu, 20 Jan 2011 16:35:48 -0800 (PST)

 Hello,
 
 I have applied changes. Please check it out.
 http://gitorious.org/run/run/trees/ratectl_fix/dev/usb/wlan
 
 
 AK
 
 

From: Juergen Lock <nox@jelal.kn-bremen.de>
To: PseudoCylon <moonlightakkiy@yahoo.ca>
Cc: bug-followup@freebsd.org, Juergen Lock <nox@jelal.kn-bremen.de>
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free
 panic
Date: Fri, 21 Jan 2011 19:21:20 +0100

 On Thu, Jan 20, 2011 at 04:35:48PM -0800, PseudoCylon wrote:
 > Hello,
 > 
 > I have applied changes. Please check it out.
 > http://gitorious.org/run/run/trees/ratectl_fix/dev/usb/wlan
 
 I added debug output again and then after a while got a deadlock [1]
 that I suspect is caused by a lor, see below.  (lock order reversal
 between "run0" and "run0_node_lock" i.e. RUN_LOCK and IEEE80211_NODE_LOCK.)
 
 It's possible this was triggered by the first DPRINTFN() in
 run_node_cleanup() (that I turned into a device_printf() and meanwhile
 have disabled, maybe it caused a taskswitch) - but in any case I'd
 say this is not safe i.e. needs to be fixed. :)
 
 [1] box stayed up but several things got stuck so in the end I had
 to drop to ddb and do a `call doadump', and fortunately this time
 the dump worked too...
 
 (kgdb) info threads 
 [...]
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   121 Thread 100418 (PID=31634: hostapd)  sched_switch (
     td=0xffffff00758633e0, newtd=0xffffff0005b40000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 [...]
   72 Thread 100064 (PID=14: usb/usbus6)  sched_switch (td=0xffffff0005c21000, 
     newtd=0xffffff0005c20ba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   71 Thread 100063 (PID=14: usb/usbus6)  sched_switch (td=0xffffff0005c213e0, 
     newtd=0xffffff00018837c0, flags=Variable "flags" is not available.
 )
 ---Type <return> to continue, or q <return> to quit---
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   70 Thread 100062 (PID=14: usb/usbus6)  sched_switch (td=0xffffff0005c217c0, 
     newtd=0xffffff0005c213e0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   69 Thread 100061 (PID=14: usb/usbus6)  sched_switch (td=0xffffff0005c21ba0, 
     newtd=0xffffff0005c217c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   68 Thread 100057 (PID=14: usb/usbus5)  sched_switch (td=0xffffff0005c25ba0, 
     newtd=0xffffff00018907c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   67 Thread 100056 (PID=14: usb/usbus5)  sched_switch (td=0xffffff0005a853e0, 
     newtd=0xffffff00018833e0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   66 Thread 100055 (PID=14: usb/usbus5)  sched_switch (td=0xffffff0005a857c0, 
     newtd=0xffffff00018907c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   65 Thread 100054 (PID=14: usb/usbus5)  sched_switch (td=0xffffff0005a85ba0, 
 ---Type <return> to continue, or q <return> to quit---
     newtd=0xffffff0005a857c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   64 Thread 100052 (PID=14: usb/usbus4)  sched_switch (td=0xffffff0005b403e0, 
     newtd=0xffffff0005a85ba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   63 Thread 100051 (PID=14: usb/usbus4)  sched_switch (td=0xffffff0005b407c0, 
     newtd=0xffffff00018833e0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   62 Thread 100050 (PID=14: usb/usbus4)  sched_switch (td=0xffffff0005b40ba0, 
     newtd=0xffffff0005b407c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   61 Thread 100049 (PID=14: usb/usbus4)  sched_switch (td=0xffffff0005b41000, 
     newtd=0xffffff0005b40ba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   60 Thread 100048 (PID=14: usb/usbus3)  sched_switch (td=0xffffff0005b413e0, 
     newtd=0xffffff0005b41000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 ---Type <return> to continue, or q <return> to quit---
   59 Thread 100047 (PID=14: usb/usbus3)  sched_switch (td=0xffffff0005b417c0, 
     newtd=0xffffff0001883000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   58 Thread 100046 (PID=14: usb/usbus3)  sched_switch (td=0xffffff0005b41ba0, 
     newtd=0xffffff0005b417c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   57 Thread 100045 (PID=14: usb/usbus3)  sched_switch (td=0xffffff0001a2cba0, 
     newtd=0xffffff0005b41ba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   56 Thread 100043 (PID=14: usb/usbus2)  sched_switch (td=0xffffff0005a813e0, 
     newtd=0xffffff00018837c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   55 Thread 100042 (PID=14: usb/usbus2)  sched_switch (td=0xffffff0005a817c0, 
     newtd=0xffffff00018907c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   54 Thread 100041 (PID=14: usb/usbus2)  sched_switch (td=0xffffff0005a81ba0, 
     newtd=0xffffff0001883000, flags=Variable "flags" is not available.
 )
 ---Type <return> to continue, or q <return> to quit---
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   53 Thread 100040 (PID=14: usb/usbus2)  sched_switch (td=0xffffff0005a83000, 
     newtd=0xffffff0005a81ba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   52 Thread 100039 (PID=14: usb/usbus1)  sched_switch (td=0xffffff0005a833e0, 
     newtd=0xffffff00018907c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   51 Thread 100038 (PID=14: usb/usbus1)  sched_switch (td=0xffffff0005a837c0, 
     newtd=0xffffff00018837c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   50 Thread 100037 (PID=14: usb/usbus1)  sched_switch (td=0xffffff0005a83ba0, 
     newtd=0xffffff0005a837c0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   49 Thread 100036 (PID=14: usb/usbus1)  sched_switch (td=0xffffff0005a85000, 
     newtd=0xffffff0005a83ba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   48 Thread 100035 (PID=14: usb/usbus0)  sched_switch (td=0xffffff00019fe7c0, 
 ---Type <return> to continue, or q <return> to quit---
     newtd=0xffffff0005a85000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   47 Thread 100034 (PID=14: usb/usbus0)  sched_switch (td=0xffffff00019feba0, 
     newtd=0xffffff0001883000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   46 Thread 100033 (PID=14: usb/usbus0)  sched_switch (td=0xffffff0001a2a000, 
     newtd=0xffffff00019feba0, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
   45 Thread 100032 (PID=14: usb/usbus0)  sched_switch (td=0xffffff0001a2a3e0, 
     newtd=0xffffff0001a2a000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 (kgdb) thread 121
 [Switching to thread 121 (Thread 100418)]#0  sched_switch (
     td=0xffffff00758633e0, newtd=0xffffff0005b40000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 1850			cpuid = PCPU_GET(cpuid);
 (kgdb) bt
 #0  sched_switch (td=0xffffff00758633e0, newtd=0xffffff0005b40000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 #1  0xffffffff805f90ef in mi_switch (flags=259, newtd=0x0)
     at /data2v/home/nox/src-r81/src/sys/kern/kern_synch.c:449
 #2  0xffffffff80630fb6 in turnstile_wait (ts=Variable "ts" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/subr_turnstile.c:746
 #3  0xffffffff805e11c0 in _mtx_lock_sleep (m=0xffffff8000a6c330, 
     tid=18446742976169653216, opts=Variable "opts" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/kern_mutex.c:447
 #4  0xffffffff805e14b3 in _mtx_lock_flags (m=Variable "m" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/kern_mutex.c:203
 #5  0xffffffff8117839b in run_node_cleanup (ni=0xffffff8000f83000)
     at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:1719
 #6  0xffffffff806db816 in ieee80211_sta_leave (ni=0xffffff8000f83000)
     at /data2v/home/nox/src-r81/src/sys/net80211/ieee80211_node.c:834
 #7  0xffffffff806db94e in ieee80211_node_leave (ni=0xffffff8000f83000)
 ---Type <return> to continue, or q <return> to quit---
     at /data2v/home/nox/src-r81/src/sys/net80211/ieee80211_node.c:2508
 #8  0xffffffff806d2c13 in setmlme_common (vap=0xffffff013e1e2000, op=Variable "op" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/net80211/ieee80211_ioctl.c:1327
 #9  0xffffffff806d2db5 in ieee80211_ioctl_setmlme (vap=0xffffff013e1e2000, 
     ireq=Variable "ireq" is not available.
 ) at /data2v/home/nox/src-r81/src/sys/net80211/ieee80211_ioctl.c:1512
 #10 0xffffffff806d405a in ieee80211_ioctl_set80211 (vap=0xffffff013e1e2000, 
     cmd=Variable "cmd" is not available.
 ) at /data2v/home/nox/src-r81/src/sys/net80211/ieee80211_ioctl.c:2721
 #11 0xffffffff806f7b7b in in_control (so=0xffffff01e7ef3d48, cmd=2149607914, 
     data=0xffffff0007832460 "wlan0", ifp=0xffffff013e2c7800, 
     td=0xffffff00758633e0)
     at /data2v/home/nox/src-r81/src/sys/netinet/in.c:290
 #12 0xffffffff806a27b7 in ifioctl (so=0xffffff01e7ef3d48, cmd=2149607914, 
     data=0xffffff0007832460 "wlan0", td=0xffffff00758633e0)
     at /data2v/home/nox/src-r81/src/sys/net/if.c:2523
 #13 0xffffffff80632bc6 in kern_ioctl (td=0xffffff00758633e0, fd=3, 
     com=2149607914, data=0xffffff0007832460 "wlan0") at file.h:262
 #14 0xffffffff80632e0d in ioctl (td=0xffffff00758633e0, 
 ---Type <return> to continue, or q <return> to quit---
     uap=0xffffff80ee69ebf0)
     at /data2v/home/nox/src-r81/src/sys/kern/sys_generic.c:678
 #15 0xffffffff808e5407 in syscall (frame=0xffffff80ee69ec80)
     at /data2v/home/nox/src-r81/src/sys/amd64/amd64/trap.c:945
 #16 0xffffffff808cac31 in Xfast_syscall ()
     at /data2v/home/nox/src-r81/src/sys/amd64/amd64/exception.S:374
 #17 0x0000000800ca438c in ?? ()
 Previous frame inner to this frame (corrupt stack?)
 (kgdb) fr 5
 #5  0xffffffff8117839b in run_node_cleanup (ni=0xffffff8000f83000)
     at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:1719
 1719			RUN_LOCK(sc);
 (kgdb) l
 1714			wcid = rn->wcid;
 1715			/* sc_ni[0] is not used */
 1716			if (wcid != 0 && wcid <= RT2870_WCID_MAX)
 1717				sc->sc_ni[wcid] = NULL;
 1718		} else {
 1719			RUN_LOCK(sc);
 1720                    wcid = rn->wcid;
 1721                    if (wcid != 0 && wcid <= RT2870_WCID_MAX)
 1722                            sc->sc_ni[wcid] = NULL;
 1723                    RUN_UNLOCK(sc);
 (kgdb) down
 #4  0xffffffff805e14b3 in _mtx_lock_flags (m=Variable "m" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/kern_mutex.c:203
 203             _get_sleep_lock(m, curthread, opts, file, line);
 (kgdb) 
 #3  0xffffffff805e11c0 in _mtx_lock_sleep (m=0xffffff8000a6c330, 
     tid=18446742976169653216, opts=Variable "opts" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/kern_mutex.c:447
 447                     turnstile_wait(ts, mtx_owner(m), TS_EXCLUSIVE_QUEUE);
 (kgdb) p m
 $1 = (struct mtx *) 0xffffff8000a6c330
 (kgdb) p *m
 $2 = {lock_object = {lo_name = 0xffffff0005e799e0 "run0", 
     lo_flags = 16973824, lo_data = 0, lo_witness = 0x0}, 
   mtx_lock = 18446742974292827042}
 (kgdb) p m.mtx_lock & 0xfffffffffffffff
 $3 = 1152920405190122402
 (kgdb) p m.mtx_lock & 0xffffffffffffffff
 $4 = 18446742974292827042
 (kgdb) p m.mtx_lock & 0xfffffffffffffff8
 $5 = 18446742974292827040
 (kgdb) p (struct thread *)m.mtx_lock & 0xfffffffffffffff8
 Argument to arithmetic operation not a number or boolean.
 (kgdb) p (struct thread *)(m.mtx_lock & 0xfffffffffffffff8)
 $6 = (struct thread *) 0xffffff0005a81ba0
 (kgdb) thr 54
 [Switching to thread 54 (Thread 100041)]#0  sched_switch (
     td=0xffffff0005a81ba0, newtd=0xffffff0001883000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 1850                    cpuid = PCPU_GET(cpuid);
 (kgdb) bt
 #0  sched_switch (td=0xffffff0005a81ba0, newtd=0xffffff0001883000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 #1  0xffffffff805f90ef in mi_switch (flags=259, newtd=0x0)
     at /data2v/home/nox/src-r81/src/sys/kern/kern_synch.c:449
 #2  0xffffffff80630fb6 in turnstile_wait (ts=Variable "ts" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/subr_turnstile.c:746
 #3  0xffffffff805e11c0 in _mtx_lock_sleep (m=0xffffff8000a717c8, 
     tid=18446742974292827040, opts=Variable "opts" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/kern_mutex.c:447
 #4  0xffffffff806dad00 in ieee80211_free_node (ni=0xffffff8000f83000)
     at /data2v/home/nox/src-r81/src/sys/net80211/ieee80211_node.c:1682
 #5  0xffffffff81172e1a in run_tx_free (pq=0xffffff8000a6c350, 
     data=0xffffff8000a6c660, txerr=Variable "txerr" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2759
 #6  0xffffffff8117783d in run_bulk_tx_callbackN (xfer=0xffffff8000d1e148, 
     error=USB_ERR_NORMAL_COMPLETION, index=0)
 ---Type <return> to continue, or q <return> to quit---
     at /data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:2793
 #7  0xffffffff8052a92d in usbd_callback_wrapper (pq=Variable "pq" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/dev/usb/usb_transfer.c:2136
 #8  0xffffffff80526fa6 in usb_command_wrapper (pq=0xffffff8000d1e060, xfer=Variable "xfer" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/dev/usb/usb_transfer.c:2745
 #9  0xffffffff80529a70 in usb_callback_proc (_pm=Variable "_pm" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/dev/usb/usb_transfer.c:2005
 #10 0xffffffff80524633 in usb_process (arg=Variable "arg" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/dev/usb/usb_process.c:166
 #11 0xffffffff805c64a8 in fork_exit (
     callout=0xffffffff80524560 <usb_process>, arg=0xffffff80003e8d10, 
     frame=0xffffff80e97efc80)
     at /data2v/home/nox/src-r81/src/sys/kern/kern_fork.c:844
 #12 0xffffffff808cae2e in fork_trampoline ()
     at /data2v/home/nox/src-r81/src/sys/amd64/amd64/exception.S:562
 #13 0x0000000000000000 in ?? ()
 ---Type <return> to continue, or q <return> to quit---
 #14 0x0000000000000000 in ?? ()
 #15 0x0000000000000001 in ?? ()
 #16 0x0000000000000000 in ?? ()
 #17 0x0000000000000000 in ?? ()
 #18 0x0000000000000000 in ?? ()
 #19 0x0000000000000000 in ?? ()
 #20 0x0000000000000000 in ?? ()
 #21 0x0000000000000000 in ?? ()
 #22 0x0000000000000000 in ?? ()
 #23 0x0000000000000000 in ?? ()
 #24 0x0000000000000000 in ?? ()
 #25 0x0000000000000000 in ?? ()
 #26 0x0000000000000000 in ?? ()
 #27 0x0000000000000000 in ?? ()
 #28 0x0000000000000000 in ?? ()
 #29 0x0000000000000000 in ?? ()
 #30 0x0000000000000000 in ?? ()
 ---Type <return> to continue, or q <return> to quit---q
 Quit
 (kgdb) fr 3
 #3  0xffffffff805e11c0 in _mtx_lock_sleep (m=0xffffff8000a717c8, 
     tid=18446742974292827040, opts=Variable "opts" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/kern_mutex.c:447
 447                     turnstile_wait(ts, mtx_owner(m), TS_EXCLUSIVE_QUEUE);
 (kgdb) p m
 $7 = (struct mtx *) 0xffffff8000a717c8
 (kgdb) p *m
 $8 = {lock_object = {lo_name = 0xffffff8000a717b8 "run0_node_lock", 
     lo_flags = 17498112, lo_data = 0, lo_witness = 0x0}, 
   mtx_lock = 18446742976169653218}
 (kgdb) p (struct thread *)(m.mtx_lock & 0xfffffffffffffff8)
 $9 = (struct thread *) 0xffffff00758633e0
 (kgdb) thread 121
 [Switching to thread 121 (Thread 100418)]#0  sched_switch (
     td=0xffffff00758633e0, newtd=0xffffff0005b40000, flags=Variable "flags" is not available.
 )
     at /data2v/home/nox/src-r81/src/sys/kern/sched_ule.c:1850
 1850                    cpuid = PCPU_GET(cpuid);
 (kgdb) q

From: PseudoCylon <moonlightakkiy@yahoo.ca>
To: bug-followup@freebsd.org, Juergen Lock <nox@jelal.kn-bremen.de>
Cc: Juergen Lock <nox@jelal.kn-bremen.de>
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free panic
Date: Sat, 22 Jan 2011 23:35:14 -0800 (PST)

 ----- Original Message ----
 > From: Juergen Lock <nox@jelal.kn-bremen.de>
 > To: PseudoCylon <moonlightakkiy@yahoo.ca>
 > Cc: bug-followup@freebsd.org; Juergen Lock <nox@jelal.kn-bremen.de>
 > Sent: Fri, January 21, 2011 11:21:20 AM
 > Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free 
 >panic
 > 
 > It's possible  this was triggered by the first DPRINTFN() in
 > run_node_cleanup() (that I  turned into a device_printf() and meanwhile
 > have disabled, maybe it caused a  taskswitch)
 
 Your bt says no.
 
 > #5  0xffffffff8117839b in run_node_cleanup (ni=0xffffff8000f83000)
 >     at 
 >/data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:1719
 >
 > 1719            RUN_LOCK(sc);
 > (kgdb) l
 
 
 run_node_cleanup() was called with node lock held. Happens all the time.
 
 > - but in any case I'd
 > say this is not safe i.e. needs to be  fixed. :)
 > 
 
 Yes. Here is fix. This one shall work.
 http://gitorious.org/run/run/trees/fifo_fix/dev/usb/wlan
 
 
 AK
 
 
 

From: Juergen Lock <nox@jelal.kn-bremen.de>
To: PseudoCylon <moonlightakkiy@yahoo.ca>
Cc: bug-followup@freebsd.org, Juergen Lock <nox@jelal.kn-bremen.de>
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free
 panic
Date: Sun, 30 Jan 2011 22:50:42 +0100

 On Sat, Jan 22, 2011 at 11:35:14PM -0800, PseudoCylon wrote:
 
 > >panic
 > > 
 > > It's possible  this was triggered by the first DPRINTFN() in
 > > run_node_cleanup() (that I  turned into a device_printf() and meanwhile
 > > have disabled, maybe it caused a  taskswitch)
 > 
 > Your bt says no.
 > 
 I was more thinking the printf might have allowed the other
 thread to run and grab the lock...
 
 > > #5  0xffffffff8117839b in run_node_cleanup (ni=0xffffff8000f83000)
 > >     at 
 > >/data2v/home/nox/src-r81/src/sys/modules/usb/run/../../../dev/usb/wlan/if_run.c:1719
 > >
 > > 1719            RUN_LOCK(sc);
 > > (kgdb) l
 > 
 > 
 > run_node_cleanup() was called with node lock held. Happens all the time.
 > 
  Ok but this time RUN_LOCK was held by the same thread that slept on the
 node lock and thus there was deadlock...
 
 > > - but in any case I'd
 > > say this is not safe i.e. needs to be  fixed. :)
 > > 
 > 
 > Yes. Here is fix. This one shall work.
 > http://gitorious.org/run/run/trees/fifo_fix/dev/usb/wlan
 
  Anyway, I have been testing this version for maybe a week now and it
 seems to work at least no worse than the previous one, minus the
 deadlock. :)  So it probably can go in.
 
  Thanx!
 	Juergen

From: PseudoCylon <moonlightakkiy@yahoo.ca>
To: Juergen Lock <nox@jelal.kn-bremen.de>
Cc: bug-followup@freebsd.org, Juergen Lock <nox@jelal.kn-bremen.de>
Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free panic
Date: Mon, 31 Jan 2011 15:28:25 -0800 (PST)

 ----- Original Message ----
 > From: Juergen Lock <nox@jelal.kn-bremen.de>
 > To: PseudoCylon <moonlightakkiy@yahoo.ca>
 > Cc: bug-followup@freebsd.org; Juergen Lock <nox@jelal.kn-bremen.de>
 > Sent: Sun, January 30, 2011 2:50:42 PM
 > Subject: Re: kern/153938: [run] [panic] [patch] Workaround for use-after-free 
 >panic
 > 
 > On Sat, Jan 22, 2011 at 11:35:14PM -0800, PseudoCylon wrote:
 > 
 > 
 >  Anyway, I  have been testing this version for maybe a week now and it
 > seems to work at  least no worse than the previous one, minus the
 > deadlock. :)  So it  probably can go in.
 > 
 >  Thanx!
 >     Juergen
 > 
 
 Thanks for testing. I'll submit the patch.
 
 
 AK
 
 
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/153938: commit references a PR
Date: Mon, 14 Feb 2011 08:14:15 +0000 (UTC)

 Author: hselasky
 Date: Mon Feb 14 08:14:06 2011
 New Revision: 218676
 URL: http://svn.freebsd.org/changeset/base/218676
 
 Log:
   * Fix page fault caused by referring freed node.
   
   While updating Tx stats, already freed node could be referred and cause
   page fault. To avoid such panic, spool Tx stats in driver's softc. Then,
   on every ratectl interval, grab node though ieee80211_iterate_nodes() and
   update ratectl stats.
   
   * Simplify some code in run_iter_func().
   
   * Fix typo
   
   * Use memset instead of bzero (hselasky @)
   
   PR:		kern/153938
   Submitted by:	PseudoCylon <moonlightakkiy@yahoo.ca>
   Approved by:	thompsa (mentor)
 
 Modified:
   head/sys/dev/usb/wlan/if_run.c
   head/sys/dev/usb/wlan/if_runreg.h
   head/sys/dev/usb/wlan/if_runvar.h
 
 Modified: head/sys/dev/usb/wlan/if_run.c
 ==============================================================================
 --- head/sys/dev/usb/wlan/if_run.c	Mon Feb 14 08:09:02 2011	(r218675)
 +++ head/sys/dev/usb/wlan/if_run.c	Mon Feb 14 08:14:06 2011	(r218676)
 @@ -1684,7 +1684,7 @@ run_read_eeprom(struct run_softc *sc)
  	return (0);
  }
  
 -struct ieee80211_node *
 +static struct ieee80211_node *
  run_node_alloc(struct ieee80211vap *vap, const uint8_t mac[IEEE80211_ADDR_LEN])
  {
  	return malloc(sizeof (struct run_node), M_DEVBUF, M_NOWAIT | M_ZERO);
 @@ -1787,7 +1787,6 @@ run_newstate(struct ieee80211vap *vap, e
  		}
  		break;
  
 -
  	case IEEE80211_S_RUN:
  		if (!(sc->runbmap & bid)) {
  			if(sc->running++)
 @@ -2229,10 +2228,10 @@ run_drain_fifo(void *arg)
  {
  	struct run_softc *sc = arg;
  	struct ifnet *ifp = sc->sc_ifp;
 -	struct ieee80211_node *ni = sc->sc_ni[0];	/* make compiler happy */
  	uint32_t stat;
 -	int retrycnt = 0;
 +	uint16_t (*wstat)[3];
  	uint8_t wcid, mcs, pid;
 +	int8_t retry;
  
  	RUN_LOCK_ASSERT(sc, MA_OWNED);
  
 @@ -2250,31 +2249,32 @@ run_drain_fifo(void *arg)
  		    wcid == 0)
  			continue;
  
 -		ni = sc->sc_ni[wcid];
 -		if (ni->ni_rctls == NULL)
 -			continue;
 -
 -		/* update per-STA AMRR stats */
 -		if (stat & RT2860_TXQ_OK) {
 -			/*
 -			 * Check if there were retries, ie if the Tx
 -			 * success rate is different from the requested
 -			 * rate. Note that it works only because we do
 -			 * not allow rate fallback from OFDM to CCK.
 -			 */
 -			mcs = (stat >> RT2860_TXQ_MCS_SHIFT) & 0x7f;
 -			pid = (stat >> RT2860_TXQ_PID_SHIFT) & 0xf;
 -			if (mcs + 1 != pid)
 -				retrycnt = 1;
 -			ieee80211_ratectl_tx_complete(ni->ni_vap, ni,
 -			    IEEE80211_RATECTL_TX_SUCCESS,
 -			    &retrycnt, NULL);
 -		} else {
 -			retrycnt = 1;
 -			ieee80211_ratectl_tx_complete(ni->ni_vap, ni,
 -			    IEEE80211_RATECTL_TX_FAILURE,
 -			    &retrycnt, NULL);
 +		/*
 +		 * Even though each stat is Tx-complete-status like format,
 +		 * the device can poll stats. Because there is no guarantee
 +		 * that the referring node is still around when read the stats.
 +		 * So that, if we use ieee80211_ratectl_tx_update(), we will
 +		 * have hard time not to refer already freed node.
 +		 *
 +		 * To eliminate such page faults, we poll stats in softc.
 +		 * Then, update the rates later with ieee80211_ratectl_tx_update().
 +		 */
 +		wstat = &(sc->wcid_stats[wcid]);
 +		(*wstat)[RUN_TXCNT]++;
 +		if (stat & RT2860_TXQ_OK)
 +			(*wstat)[RUN_SUCCESS]++;
 +		else
  			ifp->if_oerrors++;
 +		/*
 +		 * Check if there were retries, ie if the Tx success rate is
 +		 * different from the requested rate. Note that it works only
 +		 * because we do not allow rate fallback from OFDM to CCK.
 +		 */
 +		mcs = (stat >> RT2860_TXQ_MCS_SHIFT) & 0x7f;
 +		pid = (stat >> RT2860_TXQ_PID_SHIFT) & 0xf;
 +		if ((retry = pid -1 - mcs) > 0) {
 +			(*wstat)[RUN_TXCNT] += retry;
 +			(*wstat)[RUN_RETRY] += retry;
  		}
  	}
  	DPRINTFN(3, "count=%d\n", sc->fifo_cnt);
 @@ -2290,46 +2290,51 @@ run_iter_func(void *arg, struct ieee8021
  	struct ieee80211com *ic = ni->ni_ic;
  	struct ifnet *ifp = ic->ic_ifp;
  	struct run_node *rn = (void *)ni;
 -	uint32_t sta[3];
 -	int txcnt = 0, success = 0, retrycnt = 0;
 -	int error;
 +	union run_stats sta[2];
 +	uint16_t (*wstat)[3];
 +	int txcnt, success, retrycnt, error;
 +
 +	RUN_LOCK(sc);
  
  	if (sc->rvp_cnt <= 1 && (vap->iv_opmode == IEEE80211_M_IBSS ||
  	    vap->iv_opmode == IEEE80211_M_STA)) {
 -		RUN_LOCK(sc);
 -
  		/* read statistic counters (clear on read) and update AMRR state */
  		error = run_read_region_1(sc, RT2860_TX_STA_CNT0, (uint8_t *)sta,
  		    sizeof sta);
  		if (error != 0)
 -			return;
 -
 -		DPRINTFN(3, "retrycnt=%d txcnt=%d failcnt=%d\n",
 -		    le32toh(sta[1]) >> 16, le32toh(sta[1]) & 0xffff,
 -		    le32toh(sta[0]) & 0xffff);
 +			goto fail;
  
  		/* count failed TX as errors */
 -		ifp->if_oerrors += le32toh(sta[0]) & 0xffff;
 +		ifp->if_oerrors += le16toh(sta[0].error.fail);
  
 -		retrycnt =
 -		    (le32toh(sta[0]) & 0xffff) +	/* failed TX count */
 -		    (le32toh(sta[1]) >> 16);		/* TX retransmission count */
 -
 -		txcnt =
 -		    retrycnt +
 -		    (le32toh(sta[1]) & 0xffff);		/* successful TX count */
 -
 -		success =
 -		    (le32toh(sta[1]) >> 16) +
 -		    (le32toh(sta[1]) & 0xffff);
 +		retrycnt = le16toh(sta[1].tx.retry);
 +		success = le16toh(sta[1].tx.success);
 +		txcnt = retrycnt + success + le16toh(sta[0].error.fail);
  
 -		ieee80211_ratectl_tx_update(vap, ni, &txcnt, &success,
 -		    &retrycnt);
 +		DPRINTFN(3, "retrycnt=%d success=%d failcnt=%d\n",
 +			retrycnt, success, le16toh(sta[0].error.fail));
 +	} else {
 +		wstat = &(sc->wcid_stats[RUN_AID2WCID(ni->ni_associd)]);
  
 -		RUN_UNLOCK(sc);
 +		if (wstat == &(sc->wcid_stats[0]) ||
 +		    wstat > &(sc->wcid_stats[RT2870_WCID_MAX]))
 +			goto fail;
 +
 +		txcnt = (*wstat)[RUN_TXCNT];
 +		success = (*wstat)[RUN_SUCCESS];
 +		retrycnt = (*wstat)[RUN_RETRY];
 +		DPRINTFN(3, "retrycnt=%d txcnt=%d success=%d\n",
 +		    retrycnt, txcnt, success);
 +
 +		memset(wstat, 0, sizeof(*wstat));
  	}
  
 +	ieee80211_ratectl_tx_update(vap, ni, &txcnt, &success, &retrycnt);
  	rn->amrr_ridx = ieee80211_ratectl_rate(ni, NULL, 0);
 +
 +fail:
 +	RUN_UNLOCK(sc);
 +
  	DPRINTFN(3, "ridx=%d\n", rn->amrr_ridx);
  }
  
 @@ -2345,6 +2350,8 @@ run_newassoc_cb(void *arg)
  
  	run_write_region_1(sc, RT2860_WCID_ENTRY(wcid),
  	    ni->ni_macaddr, IEEE80211_ADDR_LEN);
 +
 +	memset(&(sc->wcid_stats[wcid]), 0, sizeof(sc->wcid_stats[wcid]));
  }
  
  static void
 @@ -2384,8 +2391,6 @@ run_newassoc(struct ieee80211_node *ni, 
  	DPRINTF("new assoc isnew=%d associd=%x addr=%s\n",
  	    isnew, ni->ni_associd, ether_sprintf(ni->ni_macaddr));
  
 -	sc->sc_ni[wcid] = ni;
 -
  	for (i = 0; i < rs->rs_nrates; i++) {
  		rate = rs->rs_rates[i] & IEEE80211_RATE_VAL;
  		/* convert 802.11 rate to hardware rate index */
 
 Modified: head/sys/dev/usb/wlan/if_runreg.h
 ==============================================================================
 --- head/sys/dev/usb/wlan/if_runreg.h	Mon Feb 14 08:09:02 2011	(r218675)
 +++ head/sys/dev/usb/wlan/if_runreg.h	Mon Feb 14 08:14:06 2011	(r218676)
 @@ -1208,4 +1208,17 @@ static const struct rt2860_rate {
  	{ 30, 0x09 },	\
  	{ 31, 0x10 }
  
 +
 +union run_stats {
 +	uint32_t	raw;
 +	struct {
 +		uint16_t	fail;
 +		uint16_t	pad;
 +	} error;
 +	struct {
 +		uint16_t	success;
 +		uint16_t	retry;
 +	} tx;
 +} __aligned(4);
 +
  #endif	/* _IF_RUNREG_H_ */
 
 Modified: head/sys/dev/usb/wlan/if_runvar.h
 ==============================================================================
 --- head/sys/dev/usb/wlan/if_runvar.h	Mon Feb 14 08:09:02 2011	(r218675)
 +++ head/sys/dev/usb/wlan/if_runvar.h	Mon Feb 14 08:14:06 2011	(r218676)
 @@ -160,7 +160,10 @@ struct run_softc {
  	device_t			sc_dev;
  	struct usb_device		*sc_udev;
  	struct ifnet			*sc_ifp;
 -	struct ieee80211_node		*sc_ni[RT2870_WCID_MAX + 1];
 +	uint16_t			wcid_stats[RT2870_WCID_MAX + 1][3];
 +#define	RUN_TXCNT	0
 +#define	RUN_SUCCESS	1
 +#define	RUN_RETRY	2
  
  	int				(*sc_srom_read)(struct run_softc *,
  					    uint16_t, uint16_t *);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: eadler 
State-Changed-When: Tue Mar 1 10:10:55 EST 2011 
State-Changed-Why:  
committed in head 

http://www.freebsd.org/cgi/query-pr.cgi?pr=153938 
Responsible-Changed-From-To: freebsd-net->hselasky 
Responsible-Changed-By: eadler 
Responsible-Changed-When: Tue Mar 1 10:16:24 EST 2011 
Responsible-Changed-Why:  
Same as above 

http://www.freebsd.org/cgi/query-pr.cgi?pr=153938 
State-Changed-From-To: patched->closed 
State-Changed-By: eadler 
State-Changed-When: Sun Jul 24 22:17:20 UTC 2011 
State-Changed-Why:  

This has already been patched and MFCed 

http://www.freebsd.org/cgi/query-pr.cgi?pr=153938 
>Unformatted:
