From cattelan@mist.panasas.com  Wed Sep  5 19:37:53 2007
Return-Path: <cattelan@mist.panasas.com>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id BE49A16A417
	for <FreeBSD-gnats-submit@freebsd.org>; Wed,  5 Sep 2007 19:37:53 +0000 (UTC)
	(envelope-from cattelan@mist.panasas.com)
Received: from mist.panasas.com (ip65-44-182-162.z182-44-65.customer.algx.net [65.44.182.162])
	by mx1.freebsd.org (Postfix) with ESMTP id 695FA13C4A8
	for <FreeBSD-gnats-submit@freebsd.org>; Wed,  5 Sep 2007 19:37:53 +0000 (UTC)
	(envelope-from cattelan@mist.panasas.com)
Received: from mist.panasas.com (localhost [127.0.0.1])
	by mist.panasas.com (8.13.8/8.13.8) with ESMTP id l85Iv3wJ066533
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 5 Sep 2007 13:57:03 -0500 (CDT)
	(envelope-from cattelan@mist.panasas.com)
Received: (from cattelan@localhost)
	by mist.panasas.com (8.13.8/8.13.8/Submit) id l85Iv3J5066532;
	Wed, 5 Sep 2007 13:57:03 -0500 (CDT)
	(envelope-from cattelan)
Message-Id: <200709051857.l85Iv3J5066532@mist.panasas.com>
Date: Wed, 5 Sep 2007 13:57:03 -0500 (CDT)
From: Russell Cattelan <cattelan@thebarn.com>
Reply-To: Russell Cattelan <cattelan@thebarn.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: Recursive lock panic when w_mtx falls into DELAY	
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         116133
>Category:       kern
>Synopsis:       Recursive lock panic when w_mtx falls into DELAY
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Sep 05 19:40:01 GMT 2007
>Closed-Date:    Wed May 02 15:49:24 UTC 2012
>Last-Modified:  Wed May 02 15:49:24 UTC 2012
>Originator:     Russell Cattelan
>Release:        FreeBSD 6.2-STABLE amd64
>Organization:
>Environment:
System: FreeBSD mist 6.2-STABLE FreeBSD 6.2-STABLE #0: Mon Mar 26 14:29:16 CDT 2007 cattelan@mist:/usrX/src6/sys/amd64/compile/MIST amd64


	
>Description:
	
Set the MTX_NOWITNESS flag on the clock_lock.
Since witness uses mtx_spin_lock internally, any delay
on getting the w_mtx lock would cause mtx_spin_lock to
fall into the "adaptive" mode and call DELAY internally.
DELAY grabs the clock_lock which is also will
try to grab the w_mtx mutex trying to insert the clock_lock
into the witness data structures. The end result is a
recurse lock panic.
>How-To-Repeat:
	
>Fix:

	
Index: tango-client/src/freebsd6/src/sys/amd64/amd64/machdep.c
===================================================================
--- tango-client.orig/src/freebsd6/src/sys/amd64/amd64/machdep.c  2007-09-04 16:17:53.318140386 -0500
+++ tango-client/src/freebsd6/src/sys/amd64/amd64/machdep.c 2007-09-05 13:29:55.980183838 -0500
@@ -1211,9 +1211,14 @@ hammer_time(u_int64_t modulep, u_int64_t
   *       section, to set pcpu->ipending (etc...) properly, we
   *       must be able to get the icu lock, so it can't be
   *       under witness.
+  * clock_lock: the adaptive nature of the mutex_spin lock could
+  *           cause obtaining the w_mtx lock to fall into DELAY
+  *       which would then cause a recursive lock panic when
+  *       grabbing the clock_lock and subsequently the w_mtx
+  *           as part of the mtx_lock_spin(clock_lock) call.
   */
  mutex_init();
- mtx_init(&clock_lock, "clk", NULL, MTX_SPIN);
+ mtx_init(&clock_lock, "clk", NULL, MTX_SPIN | MTX_NOWITNESS);
  mtx_init(&icu_lock, "icu", NULL, MTX_SPIN | MTX_NOWITNESS);

  /* exceptions */
Index: tango-client/src/freebsd6/src/sys/i386/i386/machdep.c
===================================================================
--- tango-client.orig/src/freebsd6/src/sys/i386/i386/machdep.c  2007-09-03 16:21:05.000000000 -0500
+++ tango-client/src/freebsd6/src/sys/i386/i386/machdep.c 2007-09-05 13:29:40.490965402 -0500
@@ -2135,9 +2135,14 @@ init386(first)
   *       section, to set pcpu->ipending (etc...) properly, we
   *       must be able to get the icu lock, so it can't be
   *       under witness.
+  * clock_lock: the adaptive nature of the mutex_spin lock could
+  *           cause obtaining the w_mtx lock to fall into DELAY
+  *       which would then cause a recursive lock panic when
+  *       grabbing the clock_lock and subsequently the w_mtx
+  *           as part of the mtx_lock_spin(clock_lock) call.
   */
  mutex_init();
- mtx_init(&clock_lock, "clk", NULL, MTX_SPIN);
+ mtx_init(&clock_lock, "clk", NULL, MTX_SPIN | MTX_NOWITNESS);
  mtx_init(&icu_lock, "icu", NULL, MTX_SPIN | MTX_NOWITNESS);

  /* make ldt memory segments */



>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->kmacy 
Responsible-Changed-By: kmacy 
Responsible-Changed-When: Sat Nov 17 06:53:31 UTC 2007 
Responsible-Changed-Why:  

I've seen similar problems with LOCK_PROFILING enabled. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=116133 
Responsible-Changed-From-To: kmacy->freebsd-bugs 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Sun May 29 22:47:50 UTC 2011 
Responsible-Changed-Why:  
kmacy has asked for all of his PRs to be reassigned back to the pool. 

It is not clear to me whether this is still an issue or not with the 
current code.  Due to refactoring, the lock init code is now in 
sys/x86/isa/clock.c.  The lock is now marked as MTX_NOPROFILE, it's 
not clear to me if this has the same effect on the panic as 
MTX_NOWITNESS. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=116133 
State-Changed-From-To: open->feedback 
State-Changed-By: jh 
State-Changed-When: Sat Mar 10 08:53:49 UTC 2012 
State-Changed-Why:  
Is this still a problem on recent FreeBSD versions? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=116133 
State-Changed-From-To: feedback->closed 
State-Changed-By: jh 
State-Changed-When: Wed May 2 15:49:23 UTC 2012 
State-Changed-Why:  
Feedback timeout. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=116133 
>Unformatted:
