From cattelan@rose.americas.sgi.com  Mon Dec  1 13:15:50 2003
Return-Path: <cattelan@rose.americas.sgi.com>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 4D8E316A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Mon,  1 Dec 2003 13:15:50 -0800 (PST)
Received: from tolkor.sgi.com (tolkor.SGI.COM [198.149.18.6])
	by mx1.FreeBSD.org (Postfix) with ESMTP id A7DD343FB1
	for <FreeBSD-gnats-submit@freebsd.org>; Mon,  1 Dec 2003 13:15:47 -0800 (PST)
	(envelope-from cattelan@rose.americas.sgi.com)
Received: from flecktone.americas.sgi.com (flecktone.americas.sgi.com [192.48.203.135])
	by tolkor.sgi.com (8.12.9/8.12.9/linux-outbound_gateway-1.1) with ESMTP id hB1LamHc031282
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 1 Dec 2003 15:36:48 -0600
Received: from daisy-e236.americas.sgi.com (daisy-e236.americas.sgi.com [128.162.236.214])
	by flecktone.americas.sgi.com (8.12.9/8.12.9/generic_config-1.2) with ESMTP id hB1LFkP516073703
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 1 Dec 2003 15:15:46 -0600 (CST)
Received: from rose.americas.sgi.com (rose.americas.sgi.com [128.162.232.98]) by daisy-e236.americas.sgi.com (8.12.9/SGI-server-1.8) with ESMTP id hB1LFbRn346238152 for <FreeBSD-gnats-submit@freebsd.org>; Mon, 1 Dec 2003 15:15:37 -0600 (CST)
Received: from rose.americas.sgi.com (localhost [127.0.0.1])
	by rose.americas.sgi.com (8.12.10/8.12.10) with ESMTP id hB1LFQ9f028399
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 1 Dec 2003 15:15:27 -0600 (CST)
	(envelope-from cattelan@rose.americas.sgi.com)
Received: (from cattelan@localhost)
	by rose.americas.sgi.com (8.12.10/8.12.10/Submit) id hB1LFNhC028398;
	Mon, 1 Dec 2003 15:15:23 -0600 (CST)
	(envelope-from cattelan)
Message-Id: <200312012115.hB1LFNhC028398@rose.americas.sgi.com>
Date: Mon, 1 Dec 2003 15:15:23 -0600 (CST)
From: Russell Cattelan <cattelan@xfs.org>
Reply-To: Russell Cattelan <cattelan@xfs.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: vinum panics 5.1 system
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         59878
>Category:       kern
>Synopsis:       [vinum] vinum panics 5.1 system
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    le
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Dec 01 13:20:12 PST 2003
>Closed-Date:    Sat Nov 26 16:29:13 GMT 2005
>Last-Modified:  Sat Nov 26 16:29:13 GMT 2005
>Originator:     Russell Cattelan
>Release:        FreeBSD 5.1-CURRENT i386
>Organization:
>Environment:
System: FreeBSD rose 5.1-CURRENT FreeBSD 5.1-CURRENT #0: Mon Jul 14 10:51:16 CDT 2003 root@rose:/export/build/src5/sys/i386/compile/ROSE i386


>Description:
	Dual CPU FreeBSD 5.1 server
	
>How-To-Repeat:
	It's unclear exactly what is going on but bk was being run each time the
	system went down.


Debugger(c040b1f9,1000000,c0418141,c879690c,c0b8b720) at Debugger+0x55
panic(c0418141,4000,3439000,c8796938,1) at panic+0x119
kmem_malloc(c082f0b0,4000,2,c87969b4,c0388823) at kmem_malloc+0xf7
page_alloc(c0b661c0,4000,c879699b,2,c027007d) at page_alloc+0x27
slab_zalloc(c0b661c0,2,c87969d8,d89d,c87969d0) at slab_zalloc+0x133
uma_zone_slab(c0b661c0,2,c188db40,c18f4260,c1965b00) at uma_zone_slab+0xd8
uma_zalloc_bucket(c0b661c0,2,1a0d91a,c2c7972e,c19e5640) at uma_zalloc_bucket+0x19d
uma_zalloc_arg(c0b661c0,0,2,c027024d,c0b661c0) at uma_zalloc_arg+0x2c7
malloc(4000,c043bd80,2,0,c8796a98) at malloc+0x5c
MMalloc(4000,c1a4286e,10d,c405f310,0) at MMalloc+0x62
vinumstart(c3fe9228,0,c0b8b720,c8796b1c,c0228e31) at vinumstart+0x1ba
vinumstrategy(c3fe9228,0,c08791a8,40,0) at vinumstrategy+0xa6
spec_xstrategy(c1a2b5b4,c3fe9228,c8796b38,c0227f68,c8796b64) at spec_xstrategy+0x361
spec_specstrategy(c8796b64,c8796b80,c036d55d,c8796b64,3) at spec_specstrategy+0x1b
spec_vnoperate(c8796b64,3,c3fe9228,c8796b8c,c02b0247) at spec_vnoperate+0x18
ufs_strategy(c8796ba8,c8796be4,c02abd47,c8796ba8,1) at ufs_strategy+0xdd
ufs_vnoperate(c8796ba8,1,c03bf5d9,c8796bac,c026c1e4) at ufs_vnoperate+0x18
bwrite(c3fe9228,c8796c4c,c02b4a4a,c3fe9228,c3fe9358) at bwrite+0x427
bawrite(c3fe9228,c3fe9358,4,3742e0,0) at bawrite+0x1c
cluster_wbuild(c23e5b68,2000,0,0,2) at cluster_wbuild+0x7da
vfs_bio_awrite(c405dcb8,12,c0b8b720,10,0) at vfs_bio_awrite+0x233
flushbufqueues(0,c0470700,44,c040e550,64) at flushbufqueues+0x1a7
buf_daemon(0,c8796d48,0,0,0) at buf_daemon+0x13c
fork_exit(c02add60,0,c8796d48) at fork_exit+0xb0
fork_trampoline() at fork_trampoline+0x1a
--- trap 0x1, eip = 0, esp = 0xc8796d7c, ebp = 0 ---
>Fix:
>Release-Note:
>Audit-Trail:

From: Greg 'groggy' Lehey <grog@FreeBSD.org>
To: Russell Cattelan <cattelan@xfs.org>
Cc: FreeBSD-gnats-submit@FreeBSD.org
Subject: Re: kern/59878: vinum panics 5.1 system
Date: Tue, 2 Dec 2003 09:13:36 +1030

 --z0PJazCl8/BmicYQ
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 On Monday,  1 December 2003 at 15:15:23 -0600, Russell Cattelan wrote:
 >> How-To-Repeat:
 > 	It's unclear exactly what is going on but bk was being run each time the
 > 	system went down.
 
 What's bk?
 
 You don't even state the panic string.  Please supply the information
 requested in the man page and on the web site
 (http://www.vinumvm.org/vinum/how-to-debug.html):
 
     * What problems are you having?
     * Which version of FreeBSD are you running?
     * Have you made any changes to the system sources, including
       Vinum?
 
     * Supply the output of the vinum list command. If you can't start
       Vinum, supply the on-disk configuration, as described below. If
       you can't start Vinum, then (and only then) send a copy of the
       configuration file.
 
     * Supply an extract of the Vinum history file. Unless you have
       explicitly renamed it, it will be /var/log/vinum_history. This
       file can get very big; please limit it to the time around when
       you have the problems. Each line contains a timestamp at the
       beginning, so you will have no difficulty in establishing which
       data is of relevance.
 
     * Supply an extract of the file /var/log/messages. Restrict the
       extract to the same time frame as the history file. Again, each
       line contains a timestamp at the beginning, so you will have no
       difficulty in establishing which data is of relevance.
 
     * If you have a crash, please supply a backtrace from the dump
       analysis as discussed below under Kernel Panics. Please don't
       delete the crash dump; it may be needed for further analysis.
 
 What not to supply
 
 Please don't supply the following information unless I ask for it:
 
     * The output of the vinum printconfig command.
     * Your Vinum configuration file, unless your problem is that you
       can't start Vinum at all.
     * Your dmesg output.
     * Your kernel configuration file.
     * Processor dumps.
 
 In this case, it looks as if it could be a memory depletion problem.
 How much memory does the system have?
 
 Greg
 --
 See complete headers for address and phone numbers.
 
 --z0PJazCl8/BmicYQ
 Content-Type: application/pgp-signature
 Content-Disposition: inline
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.2.0 (FreeBSD)
 
 iD8DBQE/y8QYIubykFB6QiMRAqcpAKCrTqdRq5xqOj4ti7NTeIsfm207wQCgoJxb
 ua6/p7UuJvOxAsHOYJeFnIQ=
 =kHUA
 -----END PGP SIGNATURE-----
 
 --z0PJazCl8/BmicYQ--

From: Russell Cattelan <cattelan@xfs.org>
To: "Greg 'groggy' Lehey" <grog@FreeBSD.org>
Cc: Russell Cattelan <cattelan@xfs.org>,
	FreeBSD-gnats-submit@FreeBSD.org
Subject: Re: kern/59878: vinum panics 5.1 system
Date: Mon, 01 Dec 2003 22:27:20 -0600

 On Mon, 2003-12-01 at 16:43, Greg 'groggy' Lehey wrote:
 > On Monday,  1 December 2003 at 15:15:23 -0600, Russell Cattelan wrote:
 > >> How-To-Repeat:
 > > 	It's unclear exactly what is going on but bk was being run each time the
 > > 	system went down.
 > 
 > What's bk?
 BitKeeper.
 
 This vinum/ufs filesystem was from a running 4.9 system that
 was upgraded to 5.1, so I don't know if that has anything to do
 with it.
 
 The system itself is fairly straight forward dual 266 with 128meg 
 of ram.
 The kernel was compiled from the cvs tree checked out with the
 5.1 release tag.
 
 The console serial line was not connected at the time of the panic
 so the actually panic string did not show up. 
 I'll make sure to grab that the next time the panic happens.
 
 Just wanted to file this as a place holder for now since 
 this happened twice in 2 days something must be a muck.
 
 > 
 > You don't even state the panic string.  Please supply the information
 > requested in the man page and on the web site
 > (http://www.vinumvm.org/vinum/how-to-debug.html):
 > 
 >     * What problems are you having?
 >     * Which version of FreeBSD are you running?
 >     * Have you made any changes to the system sources, including
 >       Vinum?
 > 
 >     * Supply the output of the vinum list command. If you can't start
 >       Vinum, supply the on-disk configuration, as described below. If
 >       you can't start Vinum, then (and only then) send a copy of the
 >       configuration file.
 > 
 >     * Supply an extract of the Vinum history file. Unless you have
 >       explicitly renamed it, it will be /var/log/vinum_history. This
 >       file can get very big; please limit it to the time around when
 >       you have the problems. Each line contains a timestamp at the
 >       beginning, so you will have no difficulty in establishing which
 >       data is of relevance.
 > 
 >     * Supply an extract of the file /var/log/messages. Restrict the
 >       extract to the same time frame as the history file. Again, each
 >       line contains a timestamp at the beginning, so you will have no
 >       difficulty in establishing which data is of relevance.
 > 
 >     * If you have a crash, please supply a backtrace from the dump
 >       analysis as discussed below under Kernel Panics. Please don't
 >       delete the crash dump; it may be needed for further analysis.
 > 
 > What not to supply
 > 
 > Please don't supply the following information unless I ask for it:
 > 
 >     * The output of the vinum printconfig command.
 >     * Your Vinum configuration file, unless your problem is that you
 >       can't start Vinum at all.
 >     * Your dmesg output.
 >     * Your kernel configuration file.
 >     * Processor dumps.
 > 
 > In this case, it looks as if it could be a memory depletion problem.
 > How much memory does the system have?
 > 
 > Greg
 > --
 > See complete headers for address and phone numbers.
 -- 
 Russell Cattelan <cattelan@thebarn.com>
 

From: Russell Cattelan <cattelan@xfs.org>
To: "Greg 'groggy' Lehey" <grog@FreeBSD.org>
Cc: FreeBSD-gnats-submit@FreeBSD.org
Subject: Re: kern/59878: vinum panics 5.1 system
Date: Tue, 02 Dec 2003 11:44:47 -0600

 bk did it again :-)
 Panic string
 panic: kmem_malloc(16384): kmem_map too small: 54755328 total
 allocated^M^@
 cpuid = 1; lapic.id = 01000000^M^@
 Debugger("panic")^M^@
 Stopped at      Debugger+0x55:  xchgl   %ebx,in_Debugger.0^M^@
 
 
Responsible-Changed-From-To: freebsd-bugs->grog 
Responsible-Changed-By: andre 
Responsible-Changed-When: Mon Dec 22 12:48:54 PST 2003 
Responsible-Changed-Why:  
grog is better suited to handle this case than freebsd-bugs. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=59878 
State-Changed-From-To: open->feedback 
State-Changed-By: grog 
State-Changed-When: Sun Feb 29 17:56:23 PST 2004 
State-Changed-Why:  
Question to submitter. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=59878 
State-Changed-From-To: feedback->closed 
State-Changed-By: kris 
State-Changed-When: Tue Mar 9 20:50:24 PST 2004 
State-Changed-Why:  
Duplicate of 57961 

http://www.freebsd.org/cgi/query-pr.cgi?pr=59878 
State-Changed-From-To: closed->open 
State-Changed-By: grog 
State-Changed-When: Tue Mar 9 21:01:05 PST 2004 
State-Changed-Why:  
Closed by accident. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=59878 

From: Russell Cattelan <cattelan@xfs.org>
To: cattelan@xfs.org, freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/59878: vinum panics 5.1 system
Date: Sat, 13 Mar 2004 08:06:53 -0600

 Just a follow up here.
 Increasing the max users did help the situation but it apparently
 hasn't eliminated the problem.
 
 The system crashed again yesterday with this trace.
 
 
 Debugger(c073e6b4,0,c074c5c5,cb7bda58,100) at Debugger+0x55
 panic(c074c5c5,2000,5185000,cb7bda84,c3f86cc0) at panic+0x136
 kmem_malloc(c10310a0,2000,2,cb7bdad0,c06bec12) at kmem_malloc+0xf7
 page_alloc(0,2000,cb7bdac3,2,2a4ee40) at page_alloc+0x27
 uma_large_malloc(2000,2,cb7bdae4,4052555b,ccba5b76) at 
 uma_large_malloc+0x52
 malloc(2000,c0770c40,2,cb7bdb20,c4008f21) at malloc+0xcf
 MMalloc(2000,c400a6fe,10f,0,c119fe30) at MMalloc+0x62
 vinumstart(c65a7c80,0,c65a7c80,cb7bdb7c,c05c8621) at vinumstart+0x1c2
 vinumstrategy(c65a7c80,c1388a00,cb7bdba4,c053b935,c65a7c80) at 
 vinumstrategy+0xa6
 dev_strategy(c65a7c80,0,c05c9117,c65a7c80,f3c09000) at dev_strategy+0x41
 spec_xstrategy(c40c8410,c65a7c80,cb7bdbc0,c053ab88,cb7bdbec) at 
 spec_xstrategy+0x1f5
 spec_specstrategy(cb7bdbec,cb7bdc0c,c05c3574,cb7bdbec,1) at 
 spec_specstrategy+0x1b
 spec_vnoperate(cb7bdbec,1,c1388a00,c49a8280,c1388a00) at 
 spec_vnoperate+0x18
 bwrite(c65a7c80,c1388a00,cb7bdc48,80012,10) at bwrite+0x524
 vfs_bio_awrite(c65a7c80,80012,0,c1388a00,c053ab88) at 
 vfs_bio_awrite+0x291
 vop_stdfsync(cb7bdcd8,0,c40c8410,cb7bdcb0,c053ab88) at 
 vop_stdfsync+0x1c7
 spec_fsync(cb7bdcd8,cb7bdd14,c05d7777,cb7bdcd8,30002) at spec_fsync+0x30
 spec_vnoperate(cb7bdcd8,30002,c1388a00,0,0) at spec_vnoperate+0x18
 sched_sync(0,cb7bdd48,646e730d,7561755f,2e6f6964) at sched_sync+0x2d7
 fork_exit(c05d74a0,0,cb7bdd48) at fork_exit+0x80
 fork_trampoline() at fork_trampoline+0x8
 
Responsible-Changed-From-To: grog->le 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Sep 9 19:05:00 GMT 2004 
Responsible-Changed-Why:  
With permission of both, reassign from grog to le. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=59878 
State-Changed-From-To: open->closed 
State-Changed-By: le 
State-Changed-When: Sat Nov 26 16:28:35 GMT 2005 
State-Changed-Why:  
Since 'classic' vinum isn't supported anymore, close this PR. 

If the described behaviour still happens on newer systems, a new PR 
can be opened. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=59878 
>Unformatted:
