From nobody@FreeBSD.org  Mon Sep 16 04:10:03 2013
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTP id 9A4B6C04
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 16 Sep 2013 04:10:03 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 6EFFA2904
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 16 Sep 2013 04:10:03 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r8G4A3ef005748
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 16 Sep 2013 04:10:03 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r8G4A29S005719;
	Mon, 16 Sep 2013 04:10:02 GMT
	(envelope-from nobody)
Message-Id: <201309160410.r8G4A29S005719@oldred.freebsd.org>
Date: Mon, 16 Sep 2013 04:10:02 GMT
From: jb <jb.1234abcd@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: lock order reversal
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         182139
>Category:       kern
>Synopsis:       [lor] lock order reversal when trying to mount or unmount
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Sep 16 04:20:00 UTC 2013
>Closed-Date:    
>Last-Modified:  Wed Apr 16 00:40:26 UTC 2014
>Originator:     jb
>Release:        FreeBSD 10.0-CURRENT #0 r255342
>Organization:
>Environment:
FreeBSD localhost.localdomain 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r255342: Sat Sep  7 09:10:09 UTC 2013     root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  i386

>Description:
$ dmesg
..
Trying to mount root from ufs:/dev/ada0s2a [rw]...
lock order reversal:
 1st 0xc71a026c ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099
 2nd 0xe19a9be8 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:262
 3rd 0xc71845c0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2099
KDB: stack backtrace:
db_trace_self_wrapper(c117e5e0,3236323a,c117000a,528,700,...) at db_trace_self_wrapper+0x2d/frame 0xf08ba288
kdb_backtrace(c1182136,c71845c0,c11693c6,c69978e8,c118b751,...) at kdb_backtrace+0x30/frame 0xf08ba2f0
witness_checkorder(c71845c0,9,c118b751,833,c71845e0,...) at witness_checkorder+0xc8a/frame 0xf08ba340
__lockmgr_args(c71845c0,80100,c71845e0,0,0,...) at __lockmgr_args+0x83f/frame 0xf08ba418
ffs_lock(f08ba498,c69831e0,c69892f8,c69831e0,c69892f8,...) at ffs_lock+0x87/frame 0xf08ba454
VOP_LOCK1_APV(c12a6bac,f08ba498,0,c11888d2,c12baff8,...) at VOP_LOCK1_APV+0x104/frame 0xf08ba480
_vn_lock(c718458c,80100,c118b751,833,c118a9b9,...) at _vn_lock+0xa1/frame 0xf08ba4c0
vget(c718458c,80100,c711cc40,57,0,...) at vget+0x74/frame 0xf08ba4f8
vfs_hash_get(c718ad20,2739b,80000,c711cc40,f08ba5d8,...) at vfs_hash_get+0xfc/frame 0xf08ba524
ffs_vgetf(c718ad20,2739b,80000,f08ba5d8,1,...) at ffs_vgetf+0x47/frame 0xf08ba580
softdep_sync_buf(c71a0238,e19a9b90,1,0,0,...) at softdep_sync_buf+0x8ef/frame 0xf08ba5e8
ffs_syncvnode(c71a0238,1,0,c0b151e7,1,...) at ffs_syncvnode+0x287/frame 0xf08ba640
ffs_truncate(c71a0238,200,0,880,c6da7580,...) at ffs_truncate+0x6b9/frame 0xf08ba800
ufs_direnter(c71a0238,c718458c,f08ba8c0,f08babcc,0,...) at ufs_direnter+0x7ed/frame 0xf08ba880
ufs_makeinode(f08babb8,f08babcc) at ufs_makeinode+0x538/frame 0xf08ba9f8
ufs_create(f08baae0,610,c718ad30,2,2,...) at ufs_create+0x2f/frame 0xf08baa0c
VOP_CREATE_APV(c12a6bac,f08baae0,f08babcc,f08baa70,c1aa0300,...) at VOP_CREATE_APV+0xfd/frame 0xf08baa38
vn_open_cred(f08bab70,f08babfc,180,0,c6da7580,c71eb690) at vn_open_cred+0x2e5/frame 0xf08bab08
vn_open(f08bab70,f08babfc,180,c71eb690,28823104,...) at vn_open+0x3b/frame 0xf08bab28
kern_openat(c711cc40,ffffff9c,28823104,0,100205,180) at kern_openat+0x315/frame 0xf08bac20
sys_open(c711cc40,f08bacc8,1245d62e,f08bac98,c0f546b0,...) at sys_open+0x38/frame 0xf08bac40
syscall(f08bad08) at syscall+0x2de/frame 0xf08bacfc
Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf08bacfc
--- syscall (5, FreeBSD ELF32, sys_open), eip = 0x2819d307, esp = 0xbfbfdc2c, ebp = 0xbfbfdd28 ---
..



>How-To-Repeat:
on every boot.
>Fix:


>Release-Note:
>Audit-Trail:

From: J B <jb.1234abcd@gmail.com>
To: bug-followup@FreeBSD.org, jb.1234abcd@gmail.com
Cc:  
Subject: Re: kern/182139: lock order reversal
Date: Mon, 16 Sep 2013 06:50:08 +0200

 --f46d0438eb6d4a888e04e678f0c5
 Content-Type: text/plain; charset=ISO-8859-1
 
 I got similar errors, I assume after umount.
 # mount -t ext2fs /dev/ada0s6 /mnt/
 # umount /mnt
 # cat /var/log/messages
 ...
 Sep 16 06:35:41 localhost kernel: lock order reversal:
 Sep 16 06:35:41 localhost kernel: 1st 0xc8077d84 ufs (ufs) @
 /usr/src/sys/kern/vfs_mount.c:1237
 Sep 16 06:35:41 localhost kernel: 2nd 0xc8077a30 syncer (syncer) @
 /usr/src/sys/kern/vfs_subr.c:2210
 Sep 16 06:35:41 localhost kernel: KDB: stack backtrace:
 Sep 16 06:35:41 localhost kernel:
 db_trace_self_wrapper(c117e5e0,7fffff00,0,ac,0,...) at
 db_trace_self_wrapper+0x2d/frame 0xf0
 b518a0
 Sep 16 06:35:41 localhost kernel:
 kdb_backtrace(c118211d,c8077a30,c118bdc2,c69979b8,c118b751,...) at
 kdb_backtrace+0x30/frame
 0xf0b51908
 Sep 16 06:35:41 localhost kernel:
 witness_checkorder(c8077a30,9,c118b751,8a2,c8077a50,...) at
 witness_checkorder+0xc8a/frame 0
 xf0b51958
 Sep 16 06:35:41 localhost kernel:
 __lockmgr_args(c8077a30,80100,c8077a50,0,0,0,c118b751,8a2) at
 __lockmgr_args+0x83f/frame 0xf
 0b51a2c
 Sep 16 06:35:41 localhost kernel:
 vop_stdlock(f0b51aa0,246,c143bf54,c143bf6c,c143bf58,...) at
 vop_stdlock+0x4d/frame 0xf0b51a5c
 Sep 16 06:35:41 localhost kernel:
 VOP_LOCK1_APV(c128a948,f0b51aa0,c0aac8c9,c8077a50,c12baff8,...) at
 VOP_LOCK1_APV+0x104/frame 0xf0b51a88
 Sep 16 06:35:41 localhost kernel:
 _vn_lock(c80779fc,80100,c118b751,8a2,c8077a60,...) at _vn_lock+0xa1/frame
 0xf0b51ac8
 Sep 16 06:35:41 localhost kernel:
 vputx(c80b42a0,0,c118ad5b,518,c1280cc8,...) at vputx+0x219/frame 0xf0b51b10
 Sep 16 06:35:41 localhost kernel:
 dounmount(c80b42a0,8000000,c7fc3930,494,c0f50777,...) at
 dounmount+0x3d1/frame 0xf0b51b70
 Sep 16 06:35:41 localhost kernel:
 sys_unmount(c7fc3930,f0b51cc8,14,c117c645,7b3,...) at
 sys_unmount+0x3a1/frame 0xf0b51c40
 Sep 16 06:35:41 localhost kernel: syscall(f0b51d08) at syscall+0x2de/frame
 0xf0b51cfc
 Sep 16 06:35:41 localhost kernel: Xint0x80_syscall() at
 Xint0x80_syscall+0x21/frame 0xf0b51cfc
 Sep 16 06:35:41 localhost kernel: --- syscall (22, FreeBSD ELF32,
 sys_unmount), eip = 0x280c826b, esp = 0xbfbfd284, ebp = 0xbfbfd350 ---
 Sep 16 06:35:41 localhost kernel: lock order reversal:
 Sep 16 06:35:41 localhost kernel: 1st 0xc8077d84 ufs (ufs) @
 /usr/src/sys/kern/vfs_mount.c:1237
 Sep 16 06:35:41 localhost kernel: 2nd 0xc757f4a4 devfs (devfs) @
 /usr/src/sys/modules/ext2fs/../../fs/ext2fs/ext2_vfsops.c:872
 Sep 16 06:35:41 localhost kernel: KDB: stack backtrace:
 Sep 16 06:35:41 localhost kernel:
 db_trace_self_wrapper(c117e5e0,7478652f,2f736632,32747865,7366765f,...) at
 db_trace_self_wrapper+0x2d/frame 0xf0b51888
 Sep 16 06:35:41 localhost kernel:
 kdb_backtrace(c118211d,c757f4a4,c11751ac,c69977b0,c80aadb3,...) at
 kdb_backtrace+0x30/frame 0xf0b518f0
 Sep 16 06:35:41 localhost kernel:
 witness_checkorder(c757f4a4,9,c80aadb3,368,0,...) at
 witness_checkorder+0xc8a/frame 0xf0b51940
 Sep 16 06:35:41 localhost kernel:
 __lockmgr_args(c757f4a4,80400,c757f4c4,0,0,0,c80aadb3,368) at
 __lockmgr_args+0x83f/frame 0xf0b51a14
 Sep 16 06:35:41 localhost kernel:
 vop_stdlock(f0b51a88,c118b751,0,f0b51ae4,c80b42b0,...) at
 vop_stdlock+0x4d/frame 0xf0b51a44
 Sep 16 06:35:41 localhost kernel:
 VOP_LOCK1_APV(c1270d7c,f0b51a88,c80b42a0,f0b51ae4,c12baff8,...) at
 VOP_LOCK1_APV+0x104/frame 0xf0b51a70
 Sep 16 06:35:41 localhost kernel:
 _vn_lock(c757f470,80400,c80aadb3,368,f0b51b10,...) at _vn_lock+0xa1/frame
 0xf0b51ab0
 Sep 16 06:35:41 localhost kernel:
 ext2_sync(c80b42a0,1,c118ad5b,518,c1280cc8,...) at ext2_sync+0x233/frame
 0xf0b51b10
 Sep 16 06:35:41 localhost kernel:
 dounmount(c80b42a0,8000000,c7fc3930,494,c0f50777,...) at
 dounmount+0x49c/frame 0xf0b51b70
 Sep 16 06:35:41 localhost kernel:
 sys_unmount(c7fc3930,f0b51cc8,14,c117c645,7b3,...) at
 sys_unmount+0x3a1/frame 0xf0b51c40
 Sep 16 06:35:41 localhost kernel: syscall(f0b51d08) at syscall+0x2de/frame
 0xf0b51cfc
 Sep 16 06:35:41 localhost kernel: Xint0x80_syscall() at
 Xint0x80_syscall+0x21/frame 0xf0b51cfc
 Sep 16 06:35:41 localhost kernel: --- syscall (22, FreeBSD ELF32,
 sys_unmount), eip = 0x280c826b, esp = 0xbfbfd284, ebp = 0xbfbfd350 ---
 
 --f46d0438eb6d4a888e04e678f0c5--

From: Mark Linimon <linimon@lonesome.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/182139: lock order reversal
Date: Mon, 16 Sep 2013 20:27:47 -0500

 ----- Forwarded message from "John D. Hendrickson and Sara Darnell" <johnandsara2@cox.net> -----
 
 Date: Mon, 16 Sep 2013 16:30:24 -0400
 From: "John D. Hendrickson and Sara Darnell" <johnandsara2@cox.net>
 To: J B <jb.1234abcd@gmail.com>
 Cc: freebsd-bugs@FreeBSD.org
 Subject: Re: kern/182139: lock order reversal
 User-Agent: Thunderbird 2.0.0.24 (X11/20100228)
 
 i don't get that message using linux.  however i had to solve a bad
 IDE drive problem once caused by microsoft software the IDE company
 installed on it.
 
 what is your question?
 
 #1) wrong lock order is very serious: it's the whole damn disk hw and
 driver is built for that: if that fails you have dick.  lock order is
 all that is between your data and being trashed by old data before
 getting on disk (if it gets on disk, with wrong locks)
 
 #2) if mount throws messages it could be a hw error, driver problem,
 fat table mutation, etc.  it really depends you have to find out.  i'm
 saying: do you know where and when the problem is thrown?  because
 most locking is in driver code or memory caching code.
 
 for example, maybe a wrong fat table entry causes a glitch which
 causes the kernel to panic while sorting lock order - and you don't
 see or haven't read the source code to know that's what's going on.
 
 i haven't tried bsd yet because it seems so many messages point out
 serious problems recently developing (caused after bsd stable public
 release by hackers making "improvements" to bsd).  i was only going to
 try it becuase it used to be "rock solid" and "fully compiled".  bsd
 channel scares me as to how many problems and why: from any gov agent
 or company all over the world hacks into drivers? ughh.
 
 is there any management or overview of who is hacking into what parts
 of bsd these days?  or why?  i mean, does the univ. of cal. have
 professors that have keys?  or do foreigners have keys and are calling
 it "bsd" when it isn't?
 
 i know in my area as soon as gov workers invested in microsoft stock
 all unix classes closed and classes started requiring ms use (the area
 had few compared to CA to begin with).  ms moved into area and got gov
 contracts.  recently i think a few unix classes have arrived back. but
 nothing with funding or bite, ie, for e-commerce and e-gov. though
 apple does (some) of that.  apple makes progress there.  but most
 classes still require ms boxes.  all a damn scam if you ask me.
 
 well good luck.  hope some release for bsd is called stable and i get
 to try it.
 
 ----- End forwarded message -----
>Unformatted:
