From decibel@decibel.org  Sat Jun 29 07:35:53 2002
Return-Path: <decibel@decibel.org>
Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 576E237B400
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 29 Jun 2002 07:35:53 -0700 (PDT)
Received: from flake.decibel.org (flake.decibel.org [216.254.40.10])
	by mx1.FreeBSD.org (Postfix) with SMTP id 71BA043E06
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 29 Jun 2002 07:35:52 -0700 (PDT)
	(envelope-from decibel@decibel.org)
Received: (qmail 35093 invoked by uid 1001); 29 Jun 2002 14:35:51 -0000
Message-Id: <20020629143551.35089.qmail@flake.decibel.org>
Date: 29 Jun 2002 14:35:51 -0000
From: Jim C.Nasby <decibel@decibel.org>
Reply-To: Jim C.Nasby <decibel@decibel.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: vinum showing -2 drives after removing several stale objects
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         40001
>Category:       kern
>Synopsis:       [vinum] vinum showing -2 drives after removing several stale objects
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    le
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Jun 29 07:40:01 PDT 2002
>Closed-Date:    Sat Nov 26 15:15:26 GMT 2005
>Last-Modified:  Sat Nov 26 15:15:26 GMT 2005
>Originator:     Jim C. Nasby
>Release:        FreeBSD 4.6-RELEASE-p1 i386
>Organization:
>Environment:
System: FreeBSD flake.decibel.org 4.6-RELEASE-p1 FreeBSD 4.6-RELEASE-p1 #4: Wed Jun 26 22:36:41 CDT 2002 root@flake.decibel.org:/usr/obj/usr/src/sys/FLAKE1 i386


	Asus P2B-DS, dual 40G IDE, no SCSI in use
>Description:
I recently moved this machine from a 4x9G SCSI vinum array to 2x40G IDE. I left the original
configuration alone until I was certain that everything was working fine. Today I decided to
delete the old devices... I deleted the subdisks without a problem (not shown below), then I
decided to delete all the other old devices in one script. After running that script, vinum
reported -2 drives.

The system appears to be functioning normally right now, but I'm worried about what will
happen when I reboot.

>How-To-Repeat:
Here's what I ran...
root@flake:~/vinum#vinum l
2 drives:
D v0                    State: up       Device /dev/ad0s1h      Avail: 10431/39103 MB (26%)
D v1                    State: up       Device /dev/ad2s1h      Avail: 10431/39103 MB (26%)
D vinum0                State: referenced       Device  Avail: 0/0 MB
D vinum1                State: referenced       Device  Avail: 0/0 MB
D vinum2                State: referenced       Device  Avail: 0/0 MB
D vinum3                State: referenced       Device  Avail: 0/0 MB

15 volumes:
V mirror1a              State: up       Plexes:       2 Size:          0  B
V mirror1b              State: up       Plexes:       2 Size:          0  B
V raid10-1              State: up       Plexes:       2 Size:          0  B
V raid10-2              State: up       Plexes:       2 Size:          0  B
V raid0-1               State: up       Plexes:       1 Size:          0  B
V raid0-2               State: up       Plexes:       1 Size:          0  B
V raid5-1               State: down     Plexes:       1 Size:          0  B
V raid5-2               State: down     Plexes:       1 Size:          0  B
V raid5-3               State: down     Plexes:       1 Size:          0  B
V raid0-3               State: up       Plexes:       1 Size:          0  B
V mirror0               State: up       Plexes:       2 Size:       1024 MB
V mirror1               State: up       Plexes:       2 Size:       1024 MB
V mirror2               State: up       Plexes:       2 Size:       1024 MB
V mirror3               State: up       Plexes:       2 Size:       5120 MB
V mirror4               State: up       Plexes:       2 Size:         20 GB

24 plexes:
P mirror1a.p0         C State: up       Subdisks:     0 Size:          0  B
P mirror1a.p1         C State: faulty   Subdisks:     0 Size:          0  B
P mirror1b.p0         C State: up       Subdisks:     0 Size:          0  B
P mirror1b.p1         C State: faulty   Subdisks:     0 Size:          0  B
P raid10-1.p0         S State: up       Subdisks:     0 Size:          0  B
P raid10-1.p1         S State: faulty   Subdisks:     0 Size:          0  B
P raid10-2.p0         S State: up       Subdisks:     0 Size:          0  B
P raid10-2.p1         S State: faulty   Subdisks:     0 Size:          0  B
P raid0-1.p0          S State: up       Subdisks:     0 Size:          0  B
P raid0-2.p0          S State: up       Subdisks:     0 Size:          0  B
P raid5-1.p0         R5 State: faulty   Subdisks:     0 Size:          0  B
P raid5-2.p0         R5 State: faulty   Subdisks:     0 Size:          0  B
P raid5-3.p0         R5 State: faulty   Subdisks:     0 Size:          0  B
P raid0-3.p0          S State: up       Subdisks:     0 Size:          0  B
P mirror0.p0          C State: up       Subdisks:     1 Size:       1024 MB
P mirror0.p1          C State: faulty   Subdisks:     1 Size:       1024 MB
P mirror1.p0          C State: up       Subdisks:     1 Size:       1024 MB
P mirror1.p1          C State: up       Subdisks:     1 Size:       1024 MB
P mirror2.p0          C State: up       Subdisks:     1 Size:       1024 MB
P mirror2.p1          C State: up       Subdisks:     1 Size:       1024 MB
P mirror3.p0          C State: up       Subdisks:     1 Size:       5120 MB
P mirror3.p1          C State: up       Subdisks:     1 Size:       5120 MB
P mirror4.p0          C State: up       Subdisks:     1 Size:         20 GB
P mirror4.p1          C State: up       Subdisks:     1 Size:         20 GB

10 subdisks:
S mirror0.p0.s0         State: up       PO:        0  B Size:       1024 MB
S mirror0.p1.s0         State: stale    PO:        0  B Size:       1024 MB
S mirror1.p0.s0         State: up       PO:        0  B Size:       1024 MB
S mirror2.p0.s0         State: up       PO:        0  B Size:       1024 MB
S mirror3.p0.s0         State: up       PO:        0  B Size:       5120 MB
S mirror4.p0.s0         State: up       PO:        0  B Size:         20 GB
S mirror1.p1.s0         State: up       PO:        0  B Size:       1024 MB
S mirror2.p1.s0         State: up       PO:        0  B Size:       1024 MB
S mirror3.p1.s0         State: up       PO:        0  B Size:       5120 MB
S mirror4.p1.s0         State: up       PO:        0  B Size:         20 GB



vinum rm -f mirror1a.p0
vinum rm -f mirror1a.p1
vinum rm -f mirror1b.p0
vinum rm -f mirror1b.p1
vinum rm -f raid10-1.p0
vinum rm -f raid10-1.p1
vinum rm -f raid10-2.p0
vinum rm -f raid10-2.p1
vinum rm -f raid0-1.p0
vinum rm -f raid0-2.p0
vinum rm -f raid5-1.p0
vinum rm -f raid5-2.p0
vinum rm -f raid5-3.p0
vinum rm -f raid0-3.p0
vinum rm -f mirror1a
vinum rm -f mirror1b
vinum rm -f raid10-1
vinum rm -f raid10-2
vinum rm -f raid0-1
vinum rm -f raid0-2
vinum rm -f raid5-1
vinum rm -f raid5-2
vinum rm -f raid5-3
vinum rm -f raid0-3
vinum rm -f vinum0
vinum rm -f vinum1
vinum rm -f vinum2
vinum rm -f vinum3




-2 drives:
5 volumes:
V mirror0               State: up	Plexes:       2	Size:       1024 MB
V mirror1               State: up	Plexes:       2	Size:       1024 MB
V mirror2               State: up	Plexes:       2	Size:       1024 MB
V mirror3               State: up	Plexes:       2	Size:       5120 MB
V mirror4               State: up	Plexes:       2	Size:         20 GB

10 plexes:
P mirror0.p0          C State: up	Subdisks:     1	Size:       1024 MB
P mirror0.p1          C State: up	Subdisks:     1	Size:       1024 MB
P mirror1.p0          C State: up	Subdisks:     1	Size:       1024 MB
P mirror1.p1          C State: up	Subdisks:     1	Size:       1024 MB
P mirror2.p0          C State: up	Subdisks:     1	Size:       1024 MB
P mirror2.p1          C State: up	Subdisks:     1	Size:       1024 MB
P mirror3.p0          C State: up	Subdisks:     1	Size:       5120 MB
P mirror3.p1          C State: up	Subdisks:     1	Size:       5120 MB
P mirror4.p0          C State: up	Subdisks:     1	Size:         20 GB
P mirror4.p1          C State: up	Subdisks:     1	Size:         20 GB

10 subdisks:
S mirror0.p0.s0         State: up	PO:        0  B Size:       1024 MB
S mirror0.p1.s0         State: up	PO:        0  B Size:       1024 MB
S mirror1.p0.s0         State: up	PO:        0  B Size:       1024 MB
S mirror2.p0.s0         State: up	PO:        0  B Size:       1024 MB
S mirror3.p0.s0         State: up	PO:        0  B Size:       5120 MB
S mirror4.p0.s0         State: up	PO:        0  B Size:         20 GB
S mirror1.p1.s0         State: up	PO:        0  B Size:       1024 MB
S mirror2.p1.s0         State: up	PO:        0  B Size:       1024 MB
S mirror3.p1.s0         State: up	PO:        0  B Size:       5120 MB
S mirror4.p1.s0         State: up	PO:        0  B Size:         20 GB



Relevant section of vinum_history:

29 Jun 2002 14:15:27.378789 *** vinum started ***
29 Jun 2002 14:15:27.400686 l 
29 Jun 2002 14:15:39.058035 *** vinum started ***
29 Jun 2002 14:15:47.938778 rm -f mirror1a.p0.s0 
29 Jun 2002 14:15:52.358709 rm -f mirror1a.p1.s0 
...
Deleted first two subdisks by hand, decided that was dumb way to do it...
...
29 Jun 2002 14:16:05.926442 *** vinum started ***
29 Jun 2002 14:16:05.927934 ls 
29 Jun 2002 14:16:19.788996 *** vinum started ***
29 Jun 2002 14:16:19.790145 ls 
29 Jun 2002 14:16:45.868258 *** vinum started ***
29 Jun 2002 14:16:45.869424 ls 
29 Jun 2002 14:16:52.950559 *** vinum started ***
29 Jun 2002 14:16:52.951633 ls 
29 Jun 2002 14:17:41.468489 *** vinum started ***
29 Jun 2002 14:17:41.469593 rm -f mirror1b.p0.s0 
29 Jun 2002 14:17:41.474948 *** vinum started ***
29 Jun 2002 14:17:41.477068 rm -f mirror1b.p1.s0 
29 Jun 2002 14:17:41.481625 *** vinum started ***
29 Jun 2002 14:17:41.482704 rm -f raid10-1.p0.s0 
29 Jun 2002 14:17:41.492691 *** vinum started ***
29 Jun 2002 14:17:41.493766 rm -f raid10-1.p0.s1 
29 Jun 2002 14:17:41.497289 *** vinum started ***
29 Jun 2002 14:17:41.498320 rm -f raid10-1.p1.s0 
29 Jun 2002 14:17:41.501845 *** vinum started ***
29 Jun 2002 14:17:41.506278 rm -f raid10-1.p1.s1 
29 Jun 2002 14:17:41.509005 *** vinum started ***
29 Jun 2002 14:17:41.510014 rm -f raid10-2.p0.s0 
29 Jun 2002 14:17:41.513084 *** vinum started ***
29 Jun 2002 14:17:41.514015 rm -f raid10-2.p0.s1 
29 Jun 2002 14:17:41.520697 *** vinum started ***
29 Jun 2002 14:17:41.521771 rm -f raid10-2.p1.s0 
29 Jun 2002 14:17:41.525461 *** vinum started ***
29 Jun 2002 14:17:41.526495 rm -f raid10-2.p1.s1 
29 Jun 2002 14:17:41.530814 *** vinum started ***
29 Jun 2002 14:17:41.531807 rm -f raid0-1.p0.s0 
29 Jun 2002 14:17:41.536992 *** vinum started ***
29 Jun 2002 14:17:41.538082 rm -f raid0-1.p0.s1 
29 Jun 2002 14:17:41.542642 *** vinum started ***
29 Jun 2002 14:17:41.543685 rm -f raid0-1.p0.s2 
29 Jun 2002 14:17:41.547001 *** vinum started ***
29 Jun 2002 14:17:41.548044 rm -f raid0-1.p0.s3 
29 Jun 2002 14:17:41.554476 *** vinum started ***
29 Jun 2002 14:17:41.555455 rm -f raid0-2.p0.s0 
29 Jun 2002 14:17:41.558660 *** vinum started ***
29 Jun 2002 14:17:41.562059 rm -f raid0-2.p0.s1 
29 Jun 2002 14:17:41.567424 *** vinum started ***
29 Jun 2002 14:17:41.568444 rm -f raid0-2.p0.s2 
29 Jun 2002 14:17:41.574597 *** vinum started ***
29 Jun 2002 14:17:41.575545 rm -f raid0-2.p0.s3 
29 Jun 2002 14:17:41.582930 *** vinum started ***
29 Jun 2002 14:17:41.584012 rm -f raid5-1.p0.s0 
29 Jun 2002 14:17:41.589220 *** vinum started ***
29 Jun 2002 14:17:41.592011 rm -f raid5-1.p0.s1 
29 Jun 2002 14:17:41.600584 *** vinum started ***
29 Jun 2002 14:17:41.602740 rm -f raid5-1.p0.s2 
29 Jun 2002 14:17:41.607043 *** vinum started ***
29 Jun 2002 14:17:41.608057 rm -f raid5-1.p0.s3 
29 Jun 2002 14:17:41.615888 *** vinum started ***
29 Jun 2002 14:17:41.616923 rm -f raid5-2.p0.s0 
29 Jun 2002 14:17:41.624318 *** vinum started ***
29 Jun 2002 14:17:41.626653 rm -f raid5-2.p0.s1 
29 Jun 2002 14:17:41.634373 *** vinum started ***
29 Jun 2002 14:17:41.636341 rm -f raid5-2.p0.s2 
29 Jun 2002 14:17:41.641340 *** vinum started ***
29 Jun 2002 14:17:41.642292 rm -f raid5-2.p0.s3 
29 Jun 2002 14:17:41.649652 *** vinum started ***
29 Jun 2002 14:17:41.650679 rm -f raid5-3.p0.s0 
29 Jun 2002 14:17:41.656836 *** vinum started ***
29 Jun 2002 14:17:41.657847 rm -f raid5-3.p0.s1 
29 Jun 2002 14:17:41.665685 *** vinum started ***
29 Jun 2002 14:17:41.666787 rm -f raid5-3.p0.s2 
29 Jun 2002 14:17:41.671401 *** vinum started ***
29 Jun 2002 14:17:41.673923 rm -f raid5-3.p0.s3 
29 Jun 2002 14:17:41.677816 *** vinum started ***
29 Jun 2002 14:17:41.678771 rm -f raid0-3.p0.s0 
29 Jun 2002 14:17:41.687589 *** vinum started ***
29 Jun 2002 14:17:41.688593 rm -f raid0-3.p0.s1 
29 Jun 2002 14:17:41.694894 *** vinum started ***
29 Jun 2002 14:17:41.695909 rm -f raid0-3.p0.s2 
29 Jun 2002 14:17:41.703604 *** vinum started ***
29 Jun 2002 14:17:41.704586 rm -f raid0-3.p0.s3 
...
End of first script, everything looked fine after this (see vinum l from above>
...
29 Jun 2002 14:17:45.549374 *** vinum started ***
29 Jun 2002 14:17:45.550478 l 
29 Jun 2002 14:17:59.970521 *** vinum started ***
29 Jun 2002 14:17:59.971610 start mirror0.p1.s0 
...
Noticed stale mirror, restarted
...
29 Jun 2002 14:18:43.081578 *** vinum started ***
29 Jun 2002 14:18:43.082752 lp 
29 Jun 2002 14:18:49.335174 *** vinum started ***
29 Jun 2002 14:18:49.336404 lp 
29 Jun 2002 14:18:51.327384 *** vinum started ***
29 Jun 2002 14:18:51.328582 lp 
29 Jun 2002 14:18:55.634482 *** vinum started ***
29 Jun 2002 14:18:55.635714 lp 
29 Jun 2002 14:19:03.110201 *** vinum started ***
29 Jun 2002 14:19:03.111504 l 
29 Jun 2002 14:19:09.511237 *** vinum started ***
29 Jun 2002 14:19:09.512839 lv 
29 Jun 2002 14:19:13.715137 *** vinum started ***
29 Jun 2002 14:19:13.718209 ld 
29 Jun 2002 14:19:25.861908 *** vinum started ***
29 Jun 2002 14:19:25.863721 lp 
29 Jun 2002 14:19:31.631663 *** vinum started ***
29 Jun 2002 14:19:31.633704 lv 
29 Jun 2002 14:19:36.995931 *** vinum started ***
29 Jun 2002 14:19:36.997019 ld 
...
All the l commands to build the script
...
29 Jun 2002 14:17:59.971610 start mirror0.p1.s0 
...Not sure why this appears here...
29 Jun 2002 14:20:29.039603 *** vinum started ***
29 Jun 2002 14:20:29.040689 rm -f mirror1a.p0 
29 Jun 2002 14:20:29.045648 *** vinum started ***
29 Jun 2002 14:20:29.046799 rm -f mirror1a.p1 
29 Jun 2002 14:20:29.050263 *** vinum started ***
29 Jun 2002 14:20:29.051249 rm -f mirror1b.p0 
29 Jun 2002 14:20:29.057906 *** vinum started ***
29 Jun 2002 14:20:29.058974 rm -f mirror1b.p1 
29 Jun 2002 14:20:29.067428 *** vinum started ***
29 Jun 2002 14:20:29.068775 rm -f raid10-1.p0 
29 Jun 2002 14:20:29.074397 *** vinum started ***
29 Jun 2002 14:20:29.075894 rm -f raid10-1.p1 
29 Jun 2002 14:20:29.079337 *** vinum started ***
29 Jun 2002 14:20:29.080292 rm -f raid10-2.p0 
29 Jun 2002 14:20:29.083678 *** vinum started ***
29 Jun 2002 14:20:29.088177 rm -f raid10-2.p1 
29 Jun 2002 14:20:29.091323 *** vinum started ***
29 Jun 2002 14:20:29.092311 rm -f raid0-1.p0 
29 Jun 2002 14:20:29.095530 *** vinum started ***
29 Jun 2002 14:20:29.096537 rm -f raid0-2.p0 
29 Jun 2002 14:20:29.102865 *** vinum started ***
29 Jun 2002 14:20:29.103859 rm -f raid5-1.p0 
29 Jun 2002 14:20:29.107411 *** vinum started ***
29 Jun 2002 14:20:29.108663 rm -f raid5-2.p0 
29 Jun 2002 14:20:29.112138 *** vinum started ***
29 Jun 2002 14:20:29.113145 rm -f raid5-3.p0 
29 Jun 2002 14:20:29.117091 *** vinum started ***
29 Jun 2002 14:20:29.118106 rm -f raid0-3.p0 
29 Jun 2002 14:20:29.123366 *** vinum started ***
29 Jun 2002 14:20:29.124336 rm -f mirror1a 
29 Jun 2002 14:20:29.128188 *** vinum started ***
29 Jun 2002 14:20:29.132418 rm -f mirror1b 
29 Jun 2002 14:20:29.135579 *** vinum started ***
29 Jun 2002 14:20:29.136545 rm -f raid10-1 
29 Jun 2002 14:20:29.140284 *** vinum started ***
29 Jun 2002 14:20:29.141490 rm -f raid10-2 
29 Jun 2002 14:20:29.146446 *** vinum started ***
29 Jun 2002 14:20:29.147430 rm -f raid0-1 
29 Jun 2002 14:20:29.152498 *** vinum started ***
29 Jun 2002 14:20:29.153513 rm -f raid0-2 
29 Jun 2002 14:20:29.157032 *** vinum started ***
29 Jun 2002 14:20:29.157979 rm -f raid5-1 
29 Jun 2002 14:20:29.163972 *** vinum started ***
29 Jun 2002 14:20:29.165119 rm -f raid5-2 
29 Jun 2002 14:20:29.168746 *** vinum started ***
29 Jun 2002 14:20:29.170470 rm -f raid5-3 
29 Jun 2002 14:20:29.175963 *** vinum started ***
29 Jun 2002 14:20:29.176929 rm -f raid0-3 
29 Jun 2002 14:20:29.180659 *** vinum started ***
29 Jun 2002 14:20:29.181961 rm -f vinum0 
29 Jun 2002 14:20:29.184796 *** vinum started ***
29 Jun 2002 14:20:29.185802 rm -f vinum1 
29 Jun 2002 14:20:29.190040 *** vinum started ***
29 Jun 2002 14:20:29.191080 rm -f vinum2 
29 Jun 2002 14:20:29.195926 *** vinum started ***
29 Jun 2002 14:20:29.196891 rm -f vinum3 
...
Script done, drives now MIA
...
29 Jun 2002 14:20:31.815699 *** vinum started ***
29 Jun 2002 14:20:31.816782 l 
29 Jun 2002 14:20:51.591544 *** vinum started ***
29 Jun 2002 14:20:51.592632 l 
29 Jun 2002 14:22:43.093075 *** vinum started ***
29 Jun 2002 14:22:43.094124 l 
29 Jun 2002 14:29:55.536649 *** vinum started ***
29 Jun 2002 14:29:55.537762 l 
>Fix:
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->grog 
Responsible-Changed-By: schweikh 
Responsible-Changed-When: Wed Aug 21 01:27:03 PDT 2002 
Responsible-Changed-Why:  
Over to vinum maintainer. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40001 

From: Lukas Ertl <l.ertl@univie.ac.at>
To: freebsd-gnats-submit@FreeBSD.org, <decibel@decibel.org>
Cc:  
Subject: Re: misc/40001: vinum showing -2 drives after removing several
 staleobjects
Date: Fri, 23 Aug 2002 16:11:02 +0200 (CEST)

 I've seen this kind of "bug" too after removing stale drives, but after a
 reboot everything works fine, vinum reads the on-disk-configuration and
 shows all configured drives correctly.
 
 regards,
 le
 
 --=20
 Lukas Ertl                              eMail: l.ertl@univie.ac.at
 UNIX-Systemadministrator                Tel.:  (+43 1) 4277-14073
 Zentraler Informatikdienst (ZID)        Fax.:  (+43 1) 4277-9140
 der Universit=E4t Wien                    http://mailbox.univie.ac.at/~le/
 
Responsible-Changed-From-To: grog->le 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Sep 9 18:57:33 GMT 2004 
Responsible-Changed-Why:  
With permission of both, reassign from grog to le. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40001 
State-Changed-From-To: open->closed 
State-Changed-By: le 
State-Changed-When: Sat Nov 26 15:14:01 GMT 2005 
State-Changed-Why:  
This was a known problem in 'classic' vinum, which had some inconsistencies 
keeping its object counts up-to-date. 

Since 'classic' vinum isn't supported anymore, I'm closing this PR. 
If the described behaviour also happens on geom_vinum, a new PR can be 
opened. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=40001 
>Unformatted:
