From eugen@grosbein.pp.ru  Tue Jan  2 20:21:39 2007
Return-Path: <eugen@grosbein.pp.ru>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id A3A3316A412
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  2 Jan 2007 20:21:39 +0000 (UTC)
	(envelope-from eugen@grosbein.pp.ru)
Received: from grosbein.pp.ru (grgw.svzserv.kemerovo.su [213.184.64.166])
	by mx1.freebsd.org (Postfix) with ESMTP id 0B20B13C459
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  2 Jan 2007 20:21:38 +0000 (UTC)
	(envelope-from eugen@grosbein.pp.ru)
Received: from grosbein.pp.ru (localhost [127.0.0.1])
	by grosbein.pp.ru (8.13.8/8.13.8) with ESMTP id l02KLXAj001600
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 3 Jan 2007 03:21:33 +0700 (KRAT)
	(envelope-from eugen@grosbein.pp.ru)
Received: (from eugen@localhost)
	by grosbein.pp.ru (8.13.8/8.13.8/Submit) id l02KLXkf001599;
	Wed, 3 Jan 2007 03:21:33 +0700 (KRAT)
	(envelope-from eugen)
Message-Id: <200701022021.l02KLXkf001599@grosbein.pp.ru>
Date: Wed, 3 Jan 2007 03:21:33 +0700 (KRAT)
From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: 6.2-PRE repeatable panic: userret: Returning with 1 locks held
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         107439
>Category:       kern
>Synopsis:       [vfs] [patch] 6.2-PRE repeatable panic: userret: Returning with 1 locks held
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    kib
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 02 20:30:16 GMT 2007
>Closed-Date:    Fri Apr 09 14:35:45 UTC 2010
>Last-Modified:  Fri Apr 09 14:35:45 UTC 2010
>Originator:     Eugene Grosbein
>Release:        FreeBSD 6.2-PRERELEASE i386
>Organization:
Svyaz Service JSC
>Environment:
System: FreeBSD grosbein.pp.ru 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #7: Wed Jan 3 02:16:56 KRAT 2007 eu@grosbein.pp.ru:/mnt/home/obj/usr/local/src/sys/DADV i386
	GENERIC kernel plus options INVARIANS/INVARIANT_SUPPORT

>Description:
	An attempt to move file from r/w mounted NTFS to UFS
	produces deadlock on UFS when a kernel compiled without
	INVARIANTS or immediate panic with INVARIANTS.

>How-To-Repeat:

	I'll show how to reproduce this with file systems mounted
	using file-backed md devices; however, this problem
	exists for "real" file systems too.

	Feel free to fetch http://www.grosbein.pp.ru/panic/ntfs.img.gz
	This is compressed (152KB) image of NTFS made (8Mb) with
	Windows XP Professional Service Pack 2 (or you may use
	another NTFS if you have one).

	Then make new UFS to play with. I do not recommend to use
	with real UFS, it will be locked and clean unmount will be impossible.
	Again, you may start doing this in single mode without extra
	FS mounted and processes running.

	Now do:

dd if=/dev/zero of=ufs.img bs=1m count=1
mdufs=/dev/`mdconfig -a -t vnode -f ufs.img`
newfs $mdufs
mdntfs=/dev/`mdconfig -a -t vnode -f ntfs.img`
mkdir -p /mnt/ufs /mnt/ntfs
mount $mdufs /mnt/ufs
mount_ntfs $mdntfs /mnt/ntfs

	Now you have NTFS mounted r/w in /mnt/ntfs
	and UFS mounted r/w in /mnt/ufs. Now do:

mv /mnt/ntfs/file /mnt/ufs/

	If your kernel was compiled without INVARIANTS,
	you'll get 'Operation not supported' and the system will
	continue to run but any process trying to read from /mnt/ufs
	(including ls -l /mnt/ufs) will lock with uninterruptable disk I/O
	and will be unkillable even with kill -9.

	For the kernel with INVARIANTS (including GENERIC plus this option)
	you'll get kernel panic immediately. Sadly, crashdump always
	contains corrupted stack, was it compiled with debug info or not.
	Here is an attempt to get backtrace
	(I used 'set hw.physmem=33554432' in boot loader prompt
	or else it does not finish crashdump for all my 1024MB of RAM
	for unknown reason):

panic: userret: Returning with 1 locks held.
cpuid = 0
KDB: stack backtrace:
kdb_backtrace(c0740077,0,c07214ce,c4b19cbc,c1addc00,...) at 0xc0544a83 = kdb_backtrace+0x2f
panic(c07214ce,1,c0596fbf,c1addc00,2d,...) at 0xc0527eb1 = panic+0x129
userret(c1addc00,c4b19d38,1,280bf000,2,...) at 0xc054da0d = userret+0xf5
syscall(3b,3b,3b,bfbfee53,bfbfe8f0,...) at 0xc06e06ea = syscall+0x371
Xint0x80_syscall() at 0xc06ca1ff = Xint0x80_syscall+0x1f
--- syscall (45, FreeBSD ELF32, ktrace), eip = 0x280bf94b, esp = 0xbfbfe21c, ebp = 0xbfbfe8a8 ---
Uptime: 5m49s
Dumping 31 MB (2 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 31MB (7936 pages) 16

#0  doadump () at pcpu.h:165
165		__asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt full
#0  doadump () at pcpu.h:165
No locals.
#1  0xc0527ba0 in boot (howto=260)
    at /usr/local/src/sys/kern/kern_shutdown.c:409
	first_buf_printf = 1
#2  0xc0527f2d in panic (
    fmt=0xc07214ce "userret: Returning with %d locks held.")
    at /usr/local/src/sys/kern/kern_shutdown.c:565
	td = (struct thread *) 0xc1addc00
	bootopt = 260
	newpanic = 1
	ap = 0xc4b19cbc "\001"
	buf = "userret: Returning with 1 locks held.", '\0' <repeats 218 times>
#3  0xc054da0d in userret (td=0xc1addc00, frame=0xc4b19d38, oticks=1)
    at /usr/local/src/sys/kern/subr_trap.c:140
	p = (struct proc *) 0xc1adc430
#4  0xc06e06ea in syscall (frame=
      {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = -1077940653, tf_esi = -1077942032, tf_ebp = -1077942104, tf_isp = -994992796, tf_ebx = -1077940653, tf_edx = -1, tf_ecx = 2, tf_eax = 45, tf_trapno = 12, tf_err = 2, tf_eip = 671873355, tf_cs = 51, tf_eflags = 647, tf_esp = -1077943780, tf_ss = 59})
    at /usr/local/src/sys/i386/i386/trap.c:1034
	params = 0xbfbfe220 <Address 0xbfbfe220 out of bounds>
---Type <return> to continue, or q <return> to quit---
	callp = (struct sysent *) 0xc0756be0
	td = (struct thread *) 0xc1addc00
	p = (struct proc *) 0xc1adc430
	orig_tf_eflags = 646
	sticks = 1
	error = 45
	narg = 2
	args = {-1077940653, -1077942032, 654, 671873348, 12, 0, 1, 
  -1045576656}
	code = 128
#5  0xc06ca1ff in Xint0x80_syscall ()
    at /usr/local/src/sys/i386/i386/exception.s:200
No locals.
#6  0x00000033 in ?? ()
No symbol table info available.
Previous frame inner to this frame (corrupt stack?)

>Fix:

	Unknown. The workaround is to always mount NTFS read-only.


Eugene Grosbein
>Release-Note:
>Audit-Trail:

From: Remko Lodder <remko@FreeBSD.org>
To: Eugene Grosbein <eugen@grosbein.pp.ru>
Cc: FreeBSD-gnats-submit@FreeBSD.org
Subject: Re: kern/107439: 6.2-PRE repeatable panic: userret: Returning with
 1	locks held
Date: Tue, 02 Jan 2007 22:20:11 +0100

 Eugene Grosbein wrote:
 
 It is generally known that the NTFS code is not for
 general read/write access;
 
  From the manual page:
 
 WRITING
       There is limited writing ability.  Limitations: file must be 
 nonresident
       and must not contain any sparces (uninitialized areas); compressed 
 files
       are also not supported.  The file name must not contain multibyte 
 charac-
       ters.
 
 I think you are hitting this issue..
 
 Also please verify your issues on the mailinglists first
 before submitting a pr. It is fine thatyou want to help
 and improve the product FreeBSD, but let your issues
 expose to the world a bit more, get some discussions so
 that your PR's will be even more brilliant  :)
 
 That said; thanks for your help and willingness to improve
 FreeBSD!
 
 Cheers,
 remko
 -- 
 Kind regards,
 
       Remko Lodder               ** remko@elvandar.org
       FreeBSD                    ** remko@FreeBSD.org
 
       /* Quis custodiet ipsos custodes */
State-Changed-From-To: open->closed 
State-Changed-By: remko 
State-Changed-When: Wed Jan 3 11:12:56 UTC 2007 
State-Changed-Why:  
Lets threaten a bit. I sended you multiple follow-up messages and they 
are all being rejected by your mailserver because i am not in your 
whitelist. Since I cannot contact you at all (and you continue to submit 
reports) I am closing this one since I cannot obtain feedback from you 
unless you  are able to read my email. Thanks for the coorporation and 
your willingness to improve FreeBSD though. bye. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107439 

From: Mikolaj Golub <to.my.trociny@gmail.com>
To: bug-followup@FreeBSD.org, Eugene Grosbein <eugen@grosbein.pp.ru>
Cc:  
Subject: Re: kern/107439: 6.2-PRE repeatable panic: userret: Returning with 1 locks held
Date: Fri, 02 Apr 2010 09:07:37 +0300

 --=-=-=
 
 I have reproduced the problem on 8.0-STABLE following Eugene's
 instructions.
 
 On the system compiled without WITNESS I have:
 
 mv: rename /mnt/ntfs/test to /mnt/ufs/test: Operation not supported
 
 and any access to /mnt/ufs/test is being locked after this.
 
 On the system with WITNESS when running mv I have a witness_warn panic:
 
 System call rename returning with the following locks held:
 exclusive lockmgr ufs (ufs) r = 0 (0xc76ab058) locked @ /usr/src/sys/kern/vfs_subr.c:2091
 panic: witness_warn
 cpuid = 2
 KDB: enter: panic
 exclusive lockmgr ufs (ufs) r = 0 (0xc76ab058) locked @ /usr/src/sys/kern/vfs_subr.c:2091
 exclusive lockmgr ufs (ufs) r = 0 (0xc76ab058) locked @ /usr/src/sys/kern/vfs_subr.c:2091
 
 0xc76ab000: tag ufs, type VDIR
     usecount 1, writecount 0, refcount 3 mountedhere 0
     flags (VV_ROOT)
     v_object 0xc77637f8 ref 0 pages 1
     lock type ufs: EXCL by thread 0xc76b14a0 (pid 2100)
 #0 0xc088feb2 at __lockmgr_args+0x592
 #1 0xc0af20b1 at ffs_lock+0xa1
 #2 0xc0c047b3 at VOP_LOCK1_APV+0xf3
 #3 0xc09488d8 at _vn_lock+0x78
 #4 0xc093b98b at vget+0xbb
 #5 0xc092e7cd at vfs_hash_get+0xed
 #6 0xc0aec9d9 at ffs_vgetf+0x49
 #7 0xc0aecf4e at ffs_vget+0x2e
 #8 0xc0afc6b8 at ufs_root+0x28
 #9 0xc092fd61 at lookup+0x9a1
 #10 0xc09308bf at namei+0x5bf
 #11 0xc0944433 at kern_renameat+0x1b3
 #12 0xc0944736 at kern_rename+0x36
 #13 0xc0944769 at rename+0x29
 #14 0xc0beab50 at syscall+0x270
 #15 0xc0bcce20 at Xint0x80_syscall+0x20
         ino 2, on dev md0
 
 So we have ufs lock leakage here. The VOP_RENAME(9) routine is expected to
 vput(9) both the destination directory and file prior to returning. But for
 ntfs vop_rename() is not implemented and as a result vop_bypass() is called
 instead in VOP_RENAME_APV() and the vnods remain locked.
 
 So we need to add the vnods unlocking somewhere. I have not thought of a
 better place then in vop_rename_post() (see the patch below). The better place
 would be VOP_RENAME_APV(), after calling vop_bypass(), but VOP_RENAME_APV() is
 generated automatically by tools/vnode_if.awk and it looks like this can't be
 done there.
 
 I have tested the patch and it works for me.
 
 Also note the problem with leaked lock also exists when trying to rename
 files inside ntfs folder -- operation failed and after this you can't unmount
 ntfs. The patch cures this case too.
 
 -- 
 Mikolaj Golub
 
 
 --=-=-=
 Content-Type: text/x-patch
 Content-Disposition: inline; filename=vfs_subr.c.vop_rename.patch
 
 --- sys/kern/vfs_subr.c.orig	2010-04-02 07:42:16.000000000 +0300
 +++ sys/kern/vfs_subr.c	2010-04-02 07:58:01.000000000 +0300
 @@ -3946,6 +3946,20 @@ vop_rename_post(void *ap, int rc)
  		if (a->a_tvp)
  			VFS_KNOTE_UNLOCKED(a->a_tvp, NOTE_DELETE);
  	}
 +	/* 
 +	 * The VOP routine is expected to vput(9) both the destination
 +	 * directory and file prior to returning. If rc is EOPNOTSUPP then
 +	 * vop_rename() is not implemented for this fs and vop_bypass() has
 +	 * been called instead, so we need to unlock the vnods here.
 +	 */
 +	if (rc == EOPNOTSUPP) {
 +		if (a->a_tdvp == a->a_tvp)
 +			vrele(a->a_tdvp);
 +		else
 +			vput(a->a_tdvp);
 +		vrele(a->a_fdvp);
 +		vrele(a->a_fvp);
 +	}
  	if (a->a_tdvp != a->a_fdvp)
  		vdrop(a->a_fdvp);
  	if (a->a_tvp != a->a_fvp)
 
 --=-=-=--
State-Changed-From-To: closed->open 
State-Changed-By: linimon 
State-Changed-When: Fri Apr 2 08:49:40 UTC 2010 
State-Changed-Why:  
new patch has been received. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107439 

From: Eugene Grosbein <eugen@grosbein.pp.ru>
To: Mikolaj Golub <to.my.trociny@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/107439: 6.2-PRE repeatable panic: userret: Returning with
 1 locks held
Date: Fri, 02 Apr 2010 17:20:58 +0700

 Mikolaj Golub wrote:
 
 > I have tested the patch and it works for me.
 
 It works for me too. Thank you very much!
 
 I'll continue to run kernel compiled with this patch
 to check if it will be so stable as it was without the patch.
 
 Eugene Grosbein

From: Eugene Grosbein <egrosbein@rdtc.ru>
To: Mikolaj Golub <to.my.trociny@gmail.com>
Cc: bug-followup@FreeBSD.org, Kostik Belousov <kostikbel@gmail.com>
Subject: Re: kern/107439: 6.2-PRE repeatable panic: userret: Returning with
 1 locks held
Date: Fri, 02 Apr 2010 18:16:49 +0700

 Mikolaj Golub wrote:
 
 > I have tested the patch and it works for me.
 
 I've also found and tested another patch
 sent to freebsd-fs@ by Kostik Belousov.
 It works for me too. I'll run it for some time.
 Of course, I do not apply both patches same time.
 
 I'm copying it here for completeness.
 
 --- sys/kern/vfs_default.c.orig
 +++ sys/kern/vfs_default.c
 @@ -67,6 +67,7 @@ __FBSDID("$FreeBSD$");
  #include <vm/vnode_pager.h>
 
  static int	vop_nolookup(struct vop_lookup_args *);
 +static int	vop_norename(struct vop_rename_args *);
  static int	vop_nostrategy(struct vop_strategy_args *);
  static int	get_next_dirent(struct vnode *vp, struct dirent **dpp,
  				char *dirbuf, int dirbuflen, off_t *off,
 @@ -113,6 +114,7 @@ struct vop_vector default_vnodeops = {
  	.vop_poll =		vop_nopoll,
  	.vop_putpages =		vop_stdputpages,
  	.vop_readlink =		VOP_EINVAL,
 +	.vop_rename =		vop_norename,
  	.vop_revoke =		VOP_PANIC,
  	.vop_strategy =		vop_nostrategy,
  	.vop_unlock =		vop_stdunlock,
 @@ -206,6 +208,27 @@ vop_nolookup(ap)
  }
 
  /*
 + * vop_norename:
 + *
 + * Handle unlock and reference counting for arguments of vop_rename
 + * for filesystems that do not implement rename operation.
 + */
 +static int
 +vop_norename(struct vop_rename_args *ap)
 +{
 +
 +	if (ap->a_tvp != NULL)
 +		vput(ap->a_tvp);
 +	if (ap->a_tdvp == ap->a_tvp)
 +		vrele(ap->a_tdvp);
 +	else
 +		vput(ap->a_tdvp);
 +	vrele(ap->a_fdvp);
 +	vrele(ap->a_fvp);
 +	return (EOPNOTSUPP);
 +}
 +
 +/*
   *	vop_nostrategy:
   *
   *	Strategy routine for VFS devices that have none.
 
 
 
 Eugene Grosbein
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/107439: commit references a PR
Date: Fri,  2 Apr 2010 14:03:56 +0000 (UTC)

 Author: kib
 Date: Fri Apr  2 14:03:43 2010
 New Revision: 206094
 URL: http://svn.freebsd.org/changeset/base/206094
 
 Log:
   Supply default implementation of VOP_RENAME() that does neccessary
   unlocks and unreferences for argument vnodes, as expected by
   kern_renameat(9), and returns EOPNOTSUPP. This fixes locks and
   reference leaks when rename is attempted on fs that does not
   implement rename.
   
   PR:	kern/107439
   Based on submission by:	Mikolaj Golub <to.my.trociny gmail com>
   Tested by:	Mikolaj Golub
   MFC after:	1 week
 
 Modified:
   head/sys/kern/vfs_default.c
 
 Modified: head/sys/kern/vfs_default.c
 ==============================================================================
 --- head/sys/kern/vfs_default.c	Fri Apr  2 14:03:01 2010	(r206093)
 +++ head/sys/kern/vfs_default.c	Fri Apr  2 14:03:43 2010	(r206094)
 @@ -67,6 +67,7 @@ __FBSDID("$FreeBSD$");
  #include <vm/vnode_pager.h>
  
  static int	vop_nolookup(struct vop_lookup_args *);
 +static int	vop_norename(struct vop_rename_args *);
  static int	vop_nostrategy(struct vop_strategy_args *);
  static int	get_next_dirent(struct vnode *vp, struct dirent **dpp,
  				char *dirbuf, int dirbuflen, off_t *off,
 @@ -113,6 +114,7 @@ struct vop_vector default_vnodeops = {
  	.vop_poll =		vop_nopoll,
  	.vop_putpages =		vop_stdputpages,
  	.vop_readlink =		VOP_EINVAL,
 +	.vop_rename =		vop_norename,
  	.vop_revoke =		VOP_PANIC,
  	.vop_strategy =		vop_nostrategy,
  	.vop_unlock =		vop_stdunlock,
 @@ -206,6 +208,20 @@ vop_nolookup(ap)
  }
  
  /*
 + * vop_norename:
 + *
 + * Handle unlock and reference counting for arguments of vop_rename
 + * for filesystems that do not implement rename operation.
 + */
 +static int
 +vop_norename(struct vop_rename_args *ap)
 +{
 +
 +	vop_rename_fail(ap);
 +	return (EOPNOTSUPP);
 +}
 +
 +/*
   *	vop_nostrategy:
   *
   *	Strategy routine for VFS devices that have none.
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: kib 
State-Changed-When: Fri Apr 2 14:10:58 UTC 2010 
State-Changed-Why:  
Take. 


Responsible-Changed-From-To: freebsd-bugs->kib 
Responsible-Changed-By: kib 
Responsible-Changed-When: Fri Apr 2 14:10:58 UTC 2010 
Responsible-Changed-Why:  
Take. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107439 

From: Mark Linimon <linimon@lonesome.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/107439: 6.2-PRE repeatable panic: userret: Returning with
	1 locks held
Date: Fri, 2 Apr 2010 22:15:43 -0500

 ----- Forwarded message from Mikolaj Golub <to.my.trociny@gmail.com> -----
 
 From: Mikolaj Golub <to.my.trociny@gmail.com>
 To: Eugene Grosbein <egrosbein@rdtc.ru>
 Organization: TOA Ukraine
 Cc: Kostik Belousov <kostikbel@gmail.com>, freebsd-bugs@FreeBSD.org
 Subject: Re: kern/107439: 6.2-PRE repeatable panic: userret: Returning with
 	1 locks held
 
 The patch provided by kib@ works for me too.
 
 FreeBSD zhuzha.ua1 8.0-STABLE FreeBSD 8.0-STABLE #1: Fri Apr  2 13:47:59 EEST 2010     root@zhuzha.ua1:/usr/obj/usr/src/sys/DEBUG  i386
 
 -- 
 Mikolaj Golub
 
 ----- End forwarded message -----
State-Changed-From-To: patched->closed 
State-Changed-By: kib 
State-Changed-When: Fri Apr 9 14:35:24 UTC 2010 
State-Changed-Why:  
Merged to stable/8. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107439 
>Unformatted:
