From nobody@FreeBSD.org  Wed Oct 28 03:04:09 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id AE2101065696
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 28 Oct 2009 03:04:09 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 9B8748FC16
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 28 Oct 2009 03:04:09 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n9S3493C000921
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 28 Oct 2009 03:04:09 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n9S348OM000919;
	Wed, 28 Oct 2009 03:04:08 GMT
	(envelope-from nobody)
Message-Id: <200910280304.n9S348OM000919@www.freebsd.org>
Date: Wed, 28 Oct 2009 03:04:08 GMT
From: Ben Kaduk <kaduk@mit.edu>
To: freebsd-gnats-submit@FreeBSD.org
Subject: lock order reversal with iwn0_com_lock and iwn0 softc lock
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         140036
>Category:       kern
>Synopsis:       [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-net
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Oct 28 03:10:01 UTC 2009
>Closed-Date:    Tue Jan 12 09:52:07 UTC 2010
>Last-Modified:  Tue Jan 12 09:52:07 UTC 2010
>Originator:     Ben Kaduk
>Release:        9-curent
>Organization:
MIT SIPB
>Environment:
FreeBSD hysteresis.mit.edu 9.0-CURRENT FreeBSD 9.0-CURRENT #1: Mon Oct 26 00:21:
52 EDT 2009     kaduk@hysteresis.mit.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
With a recent -current (with the new iwn driver that has support for 5000-series
 cards, many thanks!), I get a lock order reversal on boot:
 1st 0xffffff800033d018 iwn0_com_lock (iwn0_com_lock) @ /usr/src/sys/net80211/ie
ee80211_proto.c:1082
 2nd 0xffffff8000309010 iwn0 (network driver) @ /usr/src/sys/modules/iwn/../../d
ev/iwn/if_iwn.c:3358
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
_mtx_lock_flags() at _mtx_lock_flags+0x78
iwn_wme_update() at iwn_wme_update+0xa0
ieee80211_wme_updateparams_locked() at ieee80211_wme_updateparams_locked+0xef
ieee80211_wme_updateparams() at ieee80211_wme_updateparams+0x52
ieee80211_wme_initparams() at ieee80211_wme_initparams+0x1d9
ieee80211_sta_join1() at ieee80211_sta_join1+0xb0
sta_pick_bss() at sta_pick_bss+0xb2
scan_task() at scan_task+0x594
taskqueue_run() at taskqueue_run+0x91
taskqueue_thread_loop() at taskqueue_thread_loop+0x3f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803a983d30, rbp = 0 ---

It looks like this is the same one as reported in http://lists.freebsd.org/piper
mail/freebsd-current/2009-May/007528.html
but hasn't made its way onto http://sources.zabbadoz.net/freebsd/lor.html

>How-To-Repeat:
Boot a system with iwn hardware and the driver enabled

>Fix:
The first lock is the ieee80211_com lock, and the second is the iwn softc lock.
I note that the softc lock is obtained in iwn_wme_update, which is in the call t
race after ieee80211_wme_updateparams, which obtains the ic lock.  In particular
, a quick check seems to indicate that iwn_wme_update is only ever called from i
eee80211_wme_updateparams_locked, so it seems that the ic lock should provide se
rialization, provided that it is not dropped elsewhere in ieee80211_wme_updatepa
rams_locked.  It seems that the only function (other than wme->wme_update) calle
d from ieee80211_wme_updateparams_locked is ieee80211_beacon_notify(), which in
turn calls vap->iv_update_beacon().
However, iwn does not modify vap->iv_update_beacon from the default null_update_
beacon(), which is an empty function.  Thus, it would seem that the ic lock is a
ctually sufficient protection in this case.

However, locking is not exactly my area of expertise, so I'm not entirely sure w
hether it is better to rely on the ic lock for serialization, or to drop it and
acquire the softc lock for the call to iwn_cmd() in iwn_wme_update().
I've been running my system with the former approach for about an hour, and
it seems okay with a few ssh sessions and firefox open.  With that version of th
e code, I no longer see this LOR, though I do see a different one in iwn that wa
sn't there before:
 1st 0xffffff800033d018 iwn0_com_lock (iwn0_com_lock) @ /usr/src/sys/modules/iwn
/../../dev/iwn/if_iwn.c:1735
 2nd 0xffffff8000309010 iwn0 (network driver) @ /usr/src/sys/modules/iwn/../../d
ev/iwn/if_iwn.c:3003
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
_mtx_lock_flags() at _mtx_lock_flags+0x78
iwn_raw_xmit() at iwn_raw_xmit+0x88
ieee80211_send_mgmt() at ieee80211_send_mgmt+0x4d5
sta_newstate() at sta_newstate+0x43a
ieee80211_newstate_cb() at ieee80211_newstate_cb+0xac
taskqueue_run() at taskqueue_run+0x91
taskqueue_thread_loop() at taskqueue_thread_loop+0x3f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803a983d30, rbp = 0 ---
This, however, can be eliminated by dropping the ic lock around the portion of i
wn_raw_xmit() that grabs the softc lock.  I'm now running with that version, and
it is LOR-free and doesn't seem to have had any problems during the ten minutes
I've been running it.

For what it's worth, the latter approach (drop the ic lock to pick up the softc
lock) does not seem to eliminate the original LOR, so it seems that the former a
pproach actually is better.

As such, I believe that the attached patch is correct.
( http://stuff.mit.edu/afs/sipb.mit.edu/user/kaduk/freebsd/patches/if_iwn.c.diff.2009.10.27 )


Patch attached with submission follows:

--- if_iwn.c.orig	2009-10-27 21:11:54.000000000 -0400
+++ if_iwn.c	2009-10-27 22:59:29.000000000 -0400
@@ -3000,6 +3000,7 @@
 		return ENETDOWN;
 	}
 
+	IEEE80211_UNLOCK(ic);
 	IWN_LOCK(sc);
 	if (params == NULL)
 		txq = &sc->txq[M_WME_GETAC(m)];
@@ -3025,6 +3026,7 @@
 		ifp->if_oerrors++;
 	}
 	IWN_UNLOCK(sc);
+	IEEE80211_LOCK(ic);
 	return error;
 }
 
@@ -3355,9 +3357,7 @@
 		cmd.ac[i].txoplimit =
 		    htole16(IWN_TXOP_TO_US(wmep->wmep_txopLimit));
 	}
-	IWN_LOCK(sc);
 	(void) iwn_cmd(sc, IWN_CMD_EDCA_PARAMS, &cmd, sizeof cmd, 1 /*async*/);
-	IWN_UNLOCK(sc);
 	return 0;
 #undef IWN_TXOP_TO_US
 #undef IWN_EXP2


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Oct 28 08:20:57 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=140036 

From: Bernhard Schmidt <bschmidt@techwires.net>
To: bug-followup@freebsd.org, kaduk@mit.edu
Cc:  
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock
Date: Thu, 29 Oct 2009 08:19:44 +0100

 Hi,
 
 Removing the IWN_LOCK(sc) leads to races when calling iwn_cmd(). It's better 
 to drop the IEEE80211_LOCK(ic). I do have a patch in test which hopefully will 
 hit the tree soon.
 
 --
 Bernhard

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: Bernhard Schmidt <bschmidt@techwires.net>
Cc: bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 and iwn0 softc lock
Date: Sat, 31 Oct 2009 19:24:59 -0400 (EDT)

 Indeed, I think I saw a lockup a couple days ago, possibly when
 my card had dissociated and was trying to reassociate.
 
 Thanks for looking into this,
 
 -Ben
 
 On Thu, 29 Oct 2009, Bernhard Schmidt wrote:
 
 > Hi,
 >
 > Removing the IWN_LOCK(sc) leads to races when calling iwn_cmd(). It's better
 > to drop the IEEE80211_LOCK(ic). I do have a patch in test which hopefully will
 > hit the tree soon.
 >
 > --
 > Bernhard
 >

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: Bernhard Schmidt <bschmidt@techwires.net>
Cc: bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 and iwn0 softc lock
Date: Mon, 23 Nov 2009 02:25:35 -0500 (EST)

 On Sun, 22 Nov 2009, Bernhard Schmidt wrote:
 
 > On Sunday 01 November 2009 00:24:59 you wrote:
 >> Indeed, I think I saw a lockup a couple days ago, possibly when
 >> my card had dissociated and was trying to reassociate.
 >>
 >> Thanks for looking into this,
 >
 > Seems like your initial fix for that was actually correct, that lock up you
 > saw there is related to something else. There was an issue with sending null
 > data frames.
 >
 > Can you verify that the LOR is gone with the latest checkout of my repository?
 > Compile instructions:
 > http://forums.freebsd.org/showpost.php?p=47627&postcount=16
 
 I upgraded to today's current (which picked up a number of 
 probably-unrelated changes), and then installed the driver from
 your tree on top of it.
 No LOR on boot, and I'll let you know if I see any lockups.
 
 Thanks,
 
 Ben

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: Bernhard Schmidt <bschmidt@techwires.net>
Cc: bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 and iwn0 softc lock
Date: Mon, 23 Nov 2009 21:27:15 -0500 (EST)

 On Mon, 23 Nov 2009, Benjamin Kaduk wrote:
 
 > On Sun, 22 Nov 2009, Bernhard Schmidt wrote:
 >> 
 >> Can you verify that the LOR is gone with the latest checkout of my 
 >> repository?
 >> Compile instructions:
 >> http://forums.freebsd.org/showpost.php?p=47627&postcount=16
 >
 > I upgraded to today's current (which picked up a number of probably-unrelated 
 > changes), and then installed the driver from
 > your tree on top of it.
 > No LOR on boot, and I'll let you know if I see any lockups.
 
 
 I got a "lockup" (no idea what actually was happening) while in X tonight; 
 nothing useful is in the logs.
 I'm not even sure if I can blame iwn for it ...
 
 I did get a LOR after turning on the hardware wireless switch, though:
 iwn0: RF switch: radio enabled
 wlan0: link state changed to UP
 lock order reversal:
   1st 0xffffff800033d018 iwn0_com_lock (iwn0_com_lock) @ 
 /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3280
   2nd 0xffffff8000309010 iwn0 (network driver) @ 
 /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3289
 KDB: stack backtrace:
 db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
 _witness_debugger() at _witness_debugger+0x2e
 witness_checkorder() at witness_checkorder+0x81e
 _mtx_lock_flags() at _mtx_lock_flags+0x78
 iwn_start() at iwn_start+0x35
 if_transmit() at if_transmit+0xd6
 ieee80211_start() at ieee80211_start+0x3f4
 scan_task() at scan_task+0x4c7
 taskqueue_run() at taskqueue_run+0x91
 taskqueue_thread_loop() at taskqueue_thread_loop+0x3f
 fork_exit() at fork_exit+0x12a
 fork_trampoline() at fork_trampoline+0xe
 --- trap 0, rip = 0, rsp = 0xffffff80001ffd30, rbp = 0 ---
 
 
 
 I don't think I'll have time to look at it particularly soon, though.
 
 -Ben Kaduk

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: Bernhard Schmidt <bschmidt@techwires.net>, sam@freebsd.org
Cc: bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 and iwn0 softc lock
Date: Fri, 4 Dec 2009 21:49:42 -0500 (EST)

 Okay, I've been getting these lockups fairly frequently -- in fact,
 there was this one room where I was getting them every five minutes
 or so.  I hypothesized that dissociation/association events might
 be triggers, so I set dev.iwn.0.debug=-1 and started wandering around
 this building (which is chock full of APs).
 I get:
 panic: lock (sleep mutex) iwn0_com_lock not locked @ 
 /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3254
 
 Unfortunately, it appears to have messed up my machine pretty badly,
 as only numlock blinks the LED, and I don't have a db> prompt ...
 
 Trying again without the debugging spew gets a useful debugger, though.
 "show alllocks" produces empty output (?!).
 backtrace is:
 kdb_enter
 panic
 witness_unlock
 _mtx_unlock_flags
 iwn_raw_xmit
 ieee80211_send_probereq
 beacom_miss
 taskqueue_run
 taskqueue_thread_loop
 fork_exit
 
 Looking at the coredump, I'm inclined to suspect this block of 
 ieee80211_proto.c (in beacon_miss() ):
 1.46      sam   1379:        /* XXX locking */
                  1380:        TAILQ_FOREACH(vap, &ic->ic_vaps, iv_next) {
 1.38      sam   1381:                /*
 1.46      sam   1382:                 * We only pass events through for sta vap's in RUN state;
                  1383:                 * may be too restrictive but for now this saves all the
                  1384:                 * handlers duplicating these checks.
 1.38      sam   1385:                 */
 1.46      sam   1386:                if (vap->iv_opmode == IEEE80211_M_STA &&
 1.64      sam   1387:                    vap->iv_state >= IEEE80211_S_RUN &&
 1.46      sam   1388:                    vap->iv_bmiss != NULL)
                  1389:                        vap->iv_bmiss(vap);
 
 
 Sam, do you have any thoughts as to why throwing a 
 IEEE80211_LOCK(ic)/IEEE80211_UNLOCK(ic) block around the TAILQ_FOREACH 
 might not be a good idea?
 I'm currently running with that, which gives me a new LOR
 (iwn_com_lock @ /usr/src/sys/net80211/ieee80211_scan.c:683 and
 iwn0 @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3289)
 but it hasn't locked up or panic()ed on me, yet.
 
 
 I am happy to pull more information from the coredump if needed.
 
 Thanks,
 
 Ben Kaduk

From: Bernhard Schmidt <bschmidt@techwires.net>
To: Benjamin Kaduk <kaduk@mit.edu>
Cc: sam@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock and iwn0 softc lock
Date: Sat, 5 Dec 2009 09:56:23 +0100

 On Saturday 05 December 2009 03:49:42 Benjamin Kaduk wrote:
 > Okay, I've been getting these lockups fairly frequently -- in fact,
 > there was this one room where I was getting them every five minutes
 > or so.  I hypothesized that dissociation/association events might
 > be triggers, so I set dev.iwn.0.debug=-1 and started wandering around
 > this building (which is chock full of APs).
 > I get:
 > panic: lock (sleep mutex) iwn0_com_lock not locked @
 > /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3254
 
 Are you sure your code is in sync? there is nothing accessing the lock in 
 if_iwn.c:3254
 
 3251	if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 3252		ieee80211_free_node(ni);
 3253		m_freem(m);
 3254		return ENETDOWN;
 3255	}
 
 > Unfortunately, it appears to have messed up my machine pretty badly,
 > as only numlock blinks the LED, and I don't have a db> prompt ...
 > 
 > Trying again without the debugging spew gets a useful debugger, though.
 > "show alllocks" produces empty output (?!).
 > backtrace is:
 > kdb_enter
 > panic
 > witness_unlock
 > _mtx_unlock_flags
 > iwn_raw_xmit
 > ieee80211_send_probereq
 > beacom_miss
 > taskqueue_run
 > taskqueue_thread_loop
 > fork_exit
 > 
 > Looking at the coredump, I'm inclined to suspect this block of
 > ieee80211_proto.c (in beacon_miss() ):
 > 1.46      sam   1379:        /* XXX locking */
 >                  1380:        TAILQ_FOREACH(vap, &ic->ic_vaps, iv_next) {
 > 1.38      sam   1381:                /*
 > 1.46      sam   1382:                 * We only pass events through for sta
 >  vap's in RUN state; 1383:                 * may be too restrictive but for
 >  now this saves all the 1384:                 * handlers duplicating these
 >  checks. 1.38      sam   1385:                 */
 > 1.46      sam   1386:                if (vap->iv_opmode == IEEE80211_M_STA
 >  && 1.64      sam   1387:                    vap->iv_state >=
 >  IEEE80211_S_RUN && 1.46      sam   1388:                    vap->iv_bmiss
 >  != NULL)
 >                  1389:                        vap->iv_bmiss(vap);
 > 
 > 
 > Sam, do you have any thoughts as to why throwing a
 > IEEE80211_LOCK(ic)/IEEE80211_UNLOCK(ic) block around the TAILQ_FOREACH
 > might not be a good idea?
 > I'm currently running with that, which gives me a new LOR
 > (iwn_com_lock @ /usr/src/sys/net80211/ieee80211_scan.c:683 and
 > iwn0 @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3289)
 > but it hasn't locked up or panic()ed on me, yet.
 > 
 > 
 > I am happy to pull more information from the coredump if needed.
 > 
 > Thanks,
 > 
 > Ben Kaduk
 > 
 
 -- 
 Bernhard

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: Bernhard Schmidt <bschmidt@techwires.net>
Cc: sam@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 and iwn0 softc lock
Date: Sat, 5 Dec 2009 15:25:25 -0500 (EST)

 On Sat, 5 Dec 2009, Bernhard Schmidt wrote:
 
 > On Saturday 05 December 2009 03:49:42 Benjamin Kaduk wrote:
 >> Okay, I've been getting these lockups fairly frequently -- in fact,
 >> there was this one room where I was getting them every five minutes
 >> or so.  I hypothesized that dissociation/association events might
 >> be triggers, so I set dev.iwn.0.debug=-1 and started wandering around
 >> this building (which is chock full of APs).
 >> I get:
 >> panic: lock (sleep mutex) iwn0_com_lock not locked @
 >> /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:3254
 >
 > Are you sure your code is in sync? there is nothing accessing the lock in
 > if_iwn.c:3254
 >
 > 3251	if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) {
 > 3252		ieee80211_free_node(ni);
 > 3253		m_freem(m);
 > 3254		return ENETDOWN;
 > 3255	}
 >
 
 Apparently not -- `svn info` claims that I only have r13, and
 the web the web interface shows up to r20.
 I guess I've been using cvs for too long, and expected
 `${VCS_COMMAND} diff` to default to showing differences from the
 head of the repository/current branch.
 
 I'll update and revert my change to net80211/ieee80211_proto.c and
 see what changes.
 
 Sorry for the noise,
 
 Ben

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: Bernhard Schmidt <bschmidt@techwires.net>
Cc: bug-followup@freebsd.org
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 and iwn0 softc lock
Date: Sat, 5 Dec 2009 16:30:35 -0500 (EST)

 On Sat, 5 Dec 2009, Benjamin Kaduk wrote:
 
 >
 > I'll update and revert my change to net80211/ieee80211_proto.c and
 > see what changes.
 
 Okay, now I only get a LOR on beacon miss (no panic):
 lock order reversal:
   1st 0xffffff8000309010 iwn0 (network driver) @ /usr/devel/iwn/freebsd/sys/modules/iwn/../../dev/iwn/if_iwn.c:2655
   2nd 0xffffff800033d018 iwn0_com_lock (iwn0_com_lock) @ /usr/src/sys/net80211/ieee80211_proto.c:1365
 KDB: stack backtrace:
 db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
 _witness_debugger() at _witness_debugger+0x2e
 witness_checkorder() at witness_checkorder+0x81e
 _mtx_lock_flags() at _mtx_lock_flags+0x78
 ieee80211_beacon_miss() at ieee80211_beacon_miss+0x2e
 iwn_notif_intr() at iwn_notif_intr+0x33a
 iwn_intr() at iwn_intr+0x334
 intr_event_execute_handlers() at intr_event_execute_handlers+0x66
 ithread_loop() at ithread_loop+0xb2
 fork_exit() at fork_exit+0x12a
 fork_trampoline() at fork_trampoline+0xe
 --- trap 0, rip = 0, rsp = 0xffffff803a97ed30, rbp = 0 ---
 
 I kind of expect that it's a bad idea to drop the softc lock around
 the call to ieee80211_beacom_miss() from iwn_notif_intr(), but
 don't see an easy alternative.
 
 
 More concerning, though, is the firmware error that was logged
 some twenty seconds after the beacon miss, leaving networking
 in an unuseable state:
 firmware error log:
    error type      = "SYSASSERT" (0x00000005)
    program counter = 0x00001E28
    source line     = 0x00000696
    error data      = 0x0000000100000696
    branch link     = 0x000008FA000008FA
    interrupt link  = 0x000008B200000000
    time            = 2190683577
 driver status:
    tx ring  0: qid=0  cur=99  queued=0
    tx ring  1: qid=1  cur=0   queued=0
    tx ring  2: qid=2  cur=0   queued=0
    tx ring  3: qid=3  cur=7   queued=0
    tx ring  4: qid=4  cur=186 queued=0
    tx ring  5: qid=5  cur=0   queued=0
    tx ring  6: qid=6  cur=0   queued=0
    tx ring  7: qid=7  cur=0   queued=0
    tx ring  8: qid=8  cur=0   queued=0
    tx ring  9: qid=9  cur=0   queued=0
    tx ring 10: qid=10 cur=0   queued=0
    tx ring 11: qid=11 cur=0   queued=0
    tx ring 12: qid=12 cur=0   queued=0
    tx ring 13: qid=13 cur=0   queued=0
    tx ring 14: qid=14 cur=0   queued=0
    tx ring 15: qid=15 cur=0   queued=0
    tx ring 16: qid=16 cur=0   queued=0
    tx ring 17: qid=17 cur=0   queued=0
    tx ring 18: qid=18 cur=0   queued=0
    tx ring 19: qid=19 cur=0   queued=0
    rx ring: cur=26
 
 
 Hm, maybe I should move this to not-this-ticket ...
 
 
 -Ben

From: Benjamin Kaduk <kaduk@MIT.EDU>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock
 	and iwn0 softc lock
Date: Mon, 11 Jan 2010 17:51:17 -0500 (EST)

 Helps if I complete the domain name ...
 ---------- Forwarded message ----------
 Date: Mon, 11 Jan 2010 16:53:22 -0500 (EST)
 From: Benjamin Kaduk <kaduk@mit.edu>
 To: bug-followup@mit.edu
 Cc: freebsd-net@freebsd.org
 Subject: Re: kern/140036: [iwn] [lor] lock order reversal with iwn0_com_lock 	and
       iwn0 softc lock
 
 To follow-up after a fairly long hiatus: I had been running Bernhard's
 driver from his svn repository for a while with no LORs. Over the weekend,
 I updated past FreeBSD r201882 to pick up Rui's big update from
 r201209 as well as the firmware error fix in r201882.
 I do not see the LOR anymore (and I had a rather repeatable
 test case), so I think that this PR should be closed.
 
 I didn't have a reliable way to reproduce the firmware
 error, though, so no data if that is fixed for me, too.
 
 -Ben Kaduk
 
State-Changed-From-To: open->closed 
State-Changed-By: gavin 
State-Changed-When: Tue Jan 12 09:50:39 UTC 2010 
State-Changed-Why:  
Submitter reports that this is now fixed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=140036 
>Unformatted:
