From bounces+73574-94ec-freebsd-gnats-submit=freebsd.org@sendgrid.me  Tue May 13 02:11:12 2014
Return-Path: <bounces+73574-94ec-freebsd-gnats-submit=freebsd.org@sendgrid.me>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id B5DE7A63
	for <freebsd-gnats-submit@freebsd.org>; Tue, 13 May 2014 02:11:12 +0000 (UTC)
Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85])
	by mx1.freebsd.org (Postfix) with SMTP id 5304E2E3A
	for <freebsd-gnats-submit@freebsd.org>; Tue, 13 May 2014 02:11:11 +0000 (UTC)
Received: by mf262.sendgrid.net with SMTP id mf262.38845.53717DEBA
        Tue, 13 May 2014 02:05:31 +0000 (UTC)
Received: from mail.tarsnap.com (unknown [10.100.60.108])
	by ismtpd-010 (SG) with ESMTP id 145f353de87.5129.17dfce
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 13 May 2014 02:05:31 +0000 (GMT)
Received: (qmail 5216 invoked from network); 13 May 2014 02:05:29 -0000
Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1)
  by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 13 May 2014 02:05:29 -0000
Received: (qmail 10782 invoked by uid 1001); 13 May 2014 02:03:22 -0000
Message-Id: <20140513020322.10781.qmail@clamshell.daemonology.net>
Date: 13 May 2014 02:03:22 -0000
From: Colin Percival <cperciva@freebsd.org>
Reply-To: Colin Percival <cperciva@freebsd.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: ns8250 uart spin lock held too long
X-Send-Pr-Version: 3.114
X-GNATS-Notify:

>Number:         189745
>Category:       kern
>Synopsis:       [uart] ns8250 uart spin lock held too long
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 13 02:20:00 UTC 2014
>Closed-Date:    
>Last-Modified:  Tue May 13 04:51:23 UTC 2014
>Originator:     Colin Percival
>Release:        FreeBSD 10.0-RELEASE
>Organization:
>Environment:
FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC

6 identical panics reported via panicmail; all afflicted systems
were in EC2, i.e., with hw.broken_txfifo=1.
>Description:
panic: spin lock held too long

#2  0xffffffff808af8f4 in panic (fmt=<value optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:754
#3  0xffffffff8089cb71 in _mtx_lock_spin_cookie (c=<value optimized out>, 
    tid=<value optimized out>, opts=<value optimized out>, 
    file=<value optimized out>, line=<value optimized out>)
    at /usr/src/sys/kern/kern_mutex.c:554
#4  0xffffffff80725223 in ns8250_bus_ipend (sc=0xfffff800083fc400)
    at uart_cpu.h:92
#5  0xffffffff80723fe7 in uart_intr (arg=0xfffff800083fc400) at uart_if.h:87
#6  0xffffffff80883e5b in intr_event_handle (ie=0xfffff8000819c200, 
    frame=0xfffffe03a5ff8870) at /usr/src/sys/kern/kern_intr.c:1437
#7  0xffffffff80d8d1c8 in intr_execute_handlers (isrc=0xfffff800081bf168, 
    frame=0xfffffe03a5ff8870) at /usr/src/sys/x86/x86/intr_machdep.c:269

In some panic reports there were _mtx_lock_spin_cookie -> printf -> vprintf
-> kvprintf -> putchar -> cnputs -> cnputc -> uart_cnputc cycles as a result
of the bug being re-trigerred by attempting to print the "spin lock ... held
... too long" warning.  (Which may be something worth fixing separately, by
detecting the loop and skipping straight to the panic call.)

No console output is available (panicmail does not submit it) so I don't know
which thread was holding the spin lock for so long; but it clearly unlocked
eventually since the "spinning too long -> printf -> spinning too long" cycles
were not infinite.

>How-To-Repeat:

No idea.  This panic is strongly correlated with residing in EC2, but it may
merely be caused by the broken_txfifo option since EC2 is where that option
is most often used.

>Fix:

	


>Release-Note:
>Audit-Trail:
>Unformatted:
