From nobody@FreeBSD.org  Sun Feb  3 02:07:41 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4DE1416A418
	for <freebsd-gnats-submit@FreeBSD.org>; Sun,  3 Feb 2008 02:07:41 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 3AADD13C447
	for <freebsd-gnats-submit@FreeBSD.org>; Sun,  3 Feb 2008 02:07:41 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m1325j2A002934
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 3 Feb 2008 02:05:45 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id m1325jQj002933;
	Sun, 3 Feb 2008 02:05:45 GMT
	(envelope-from nobody)
Message-Id: <200802030205.m1325jQj002933@www.freebsd.org>
Date: Sun, 3 Feb 2008 02:05:45 GMT
From: Richard Bejtlich <taosecurity@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: GEOM_CONCAT error adding second drive
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         120231
>Category:       kern
>Synopsis:       [geom] GEOM_CONCAT error adding second drive
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-geom
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 03 02:10:00 UTC 2008
>Closed-Date:    Tue Aug 11 13:53:32 UTC 2009
>Last-Modified:  Tue Nov  3 03:40:02 UTC 2009
>Originator:     Richard Bejtlich
>Release:        7.0-RC1
>Organization:
TaoSecurity
>Environment:
uname -a
FreeBSD edited.edited.com 7.0-RC1 FreeBSD 7.0-RC1 #0: Mon Dec 24 12:18:24 UTC 2007     root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386

>Description:
I want to use gconcat to make ad4s1h and ad5s1d appear to be a single
/dev/concat/nsm partition.  When ad5s1d occupies the entire ad5s1 slice,
then GEOM_CONCAT tries to add ad5s1 instead of ad5s1d.  GEOM_CONCAT
doesn't work at that point.  Please see text below for an example.
>How-To-Repeat:
# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad4s1a    989M    129M    781M    14%    /
devfs          1.0K    1.0K      0B   100%    /dev
/dev/ad4s1e    989M     22K    910M     0%    /home
/dev/ad4s1h    5.9G    4.0K    5.4G     0%    /nsm1
/dev/ad5s1d     72G    4.0K     66G     0%    /nsm2
/dev/ad4s1f    989M     12K    910M     0%    /tmp
/dev/ad4s1d    4.8G    755M    3.7G    17%    /usr
/dev/ad4s1g     20G    232K     18G     0%    /var

# mount
/dev/ad4s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad4s1e on /home (ufs, local, soft-updates)
/dev/ad4s1h on /nsm1 (ufs, local, soft-updates)
/dev/ad5s1d on /nsm2 (ufs, local, soft-updates)
/dev/ad4s1f on /tmp (ufs, local, soft-updates)
/dev/ad4s1d on /usr (ufs, local, soft-updates

# umount /nsm1
# umount /nsm2
# gconcat label -v nsm ad4s1h ad5s1d
Metadata value stored on ad4s1h.
Metadata value stored on ad5s1d.
Done.
# dmesg | tail
ad4: 38146MB <Seagate ST340014AS 8.05> at ata2-master SATA150
ad5: 76293MB <SAMSUNG HD080HJ/P ZH100-34> at ata2-slave SATA150
lapic1: Forcing LINT1 to edge trigger
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad4s1a
GEOM_CONCAT: Device nsm created (id=3517115025).
GEOM_CONCAT: Disk ad4s1h attached to nsm.
GEOM_CONCAT: Disk ad5s1 attached to nsm.
GEOM_CONCAT: Device nsm activated.
GEOM_CONCAT: Cannot add disk ad5s1d to nsm (error=17).

>Fix:
As a workaround, I can create two partitions on ad5s1 instead of one.

# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad4s1a    989M    129M    781M    14%    /
devfs          1.0K    1.0K      0B   100%    /dev
/dev/ad4s1e    989M     22K    910M     0%    /home
/dev/ad4s1h    5.9G    4.0K    5.4G     0%    /nsm1
/dev/ad5s1e     71G    4.0K     65G     0%    /nsm2
/dev/ad4s1f    989M     12K    910M     0%    /tmp
/dev/ad5s1d    989M    4.0K    910M     0%    /tmp2
/dev/ad4s1d    4.8G    755M    3.7G    17%    /usr
/dev/ad4s1g     20G    232K     18G     0%    /var

# mount
/dev/ad4s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad4s1e on /home (ufs, local, soft-updates)
/dev/ad4s1h on /nsm1 (ufs, local, soft-updates)
/dev/ad5s1e on /nsm2 (ufs, local, soft-updates)
/dev/ad4s1f on /tmp (ufs, local, soft-updates)
/dev/ad5s1d on /tmp2 (ufs, local, soft-updates)
/dev/ad4s1d on /usr (ufs, local, soft-updates)
/dev/ad4s1g on /var (ufs, local, soft-updates)

# umount /nsm1
# umount /nsm2
# gconcat label -v nsm ad4s1h ad5s1e
Metadata value stored on ad4s1h.
Metadata value stored on ad5s1e.
Done.

# dmesg | tail
acd0: DVDR <Optiarc DVD+/-RW ND-3570A/104B> at ata0-master UDMA33
ad4: 38146MB <Seagate ST340014AS 8.05> at ata2-master SATA150
ad5: 76293MB <SAMSUNG HD080HJ/P ZH100-34> at ata2-slave SATA150
lapic1: Forcing LINT1 to edge trigger
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ad4s1a
GEOM_CONCAT: Device nsm created (id=1016589974).
GEOM_CONCAT: Disk ad4s1h attached to nsm.
GEOM_CONCAT: Disk ad5s1e attached to nsm.
GEOM_CONCAT: Device nsm activated.

From here I can finish the process without a problem.

# newfs /dev/concat/nsm

# mkdir /nsm
# mount /dev/concat/nsm /nsm

# df -h
Filesystem         Size    Used   Avail Capacity  Mounted on
/dev/ad4s1a        989M    129M    781M    14%    /
devfs              1.0K    1.0K      0B   100%    /dev
/dev/ad4s1e        989M     22K    910M     0%    /home
/dev/ad4s1f        989M     12K    910M     0%    /tmp
/dev/ad5s1d        989M    4.0K    910M     0%    /tmp2
/dev/ad4s1d        4.8G    755M    3.7G    17%    /usr
/dev/ad4s1g         20G    232K     18G     0%    /var
/dev/concat/nsm     77G    4.0K     71G     0%    /nsm

# mount
/dev/ad4s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad4s1e on /home (ufs, local, soft-updates)
/dev/ad4s1f on /tmp (ufs, local, soft-updates)
/dev/ad5s1d on /tmp2 (ufs, local, soft-updates)
/dev/ad4s1d on /usr (ufs, local, soft-updates)
/dev/ad4s1g on /var (ufs, local, soft-updates)
/dev/concat/nsm on /nsm (ufs, local)

Thank you.

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-geom 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sun Feb 3 04:56:45 UTC 2008 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120231 

From: Ulf Lilleengen <lulf@freebsd.org>
To: bug-followup@FreeBSD.org, taosecurity@gmail.com
Cc:  
Subject: Re: kern/120231: [geom] GEOM_CONCAT error adding second drive
Date: Sun, 27 Jul 2008 16:34:04 +0200

 Hello,
 
 As I can see it, the reason for your problem can be that since ad5s1d is
 spanned across the whole disk, gconcat can't see the difference between ad5s1
 and ad5s1d, and thus tries to attach ad5s1 as the disk to the 'nsm' provider.
 (Error 17 as you get, means that the drive is already added). Could you
 perhaps provide bsdlabel ad5s1 output?
 
 -- 
 Ulf Lillengen

From: "Richard Bejtlich" <taosecurity@gmail.com>
To: "Ulf Lilleengen" <lulf@freebsd.org>
Cc: bug-followup@freebsd.org
Subject: Re: kern/120231: [geom] GEOM_CONCAT error adding second drive
Date: Mon, 28 Jul 2008 13:35:19 -0400

 On Sun, Jul 27, 2008 at 10:34 AM, Ulf Lilleengen <lulf@freebsd.org> wrote:
 > Hello,
 >
 > As I can see it, the reason for your problem can be that since ad5s1d is
 > spanned across the whole disk, gconcat can't see the difference between ad5s1
 > and ad5s1d, and thus tries to attach ad5s1 as the disk to the 'nsm' provider.
 > (Error 17 as you get, means that the drive is already added). Could you
 > perhaps provide bsdlabel ad5s1 output?
 >
 > --
 > Ulf Lillengen
 >
 
 Hi Ulf,
 
 I don't have that exact box available, but here's another box where I
 implemented the same workaround.
 
 # mount
 /dev/ad2s1a on / (ufs, local)
 devfs on /dev (devfs, local)
 /dev/ad2s1e on /home (ufs, local, soft-updates)
 /dev/concat/nsm on /nsm (ufs, local)
 /dev/ad2s1f on /tmp (ufs, local, soft-updates)
 /dev/ad3s1d on /tmp2 (ufs, local, soft-updates)
 /dev/ad2s1d on /usr (ufs, local, soft-updates)
 /dev/ad2s1g on /var (ufs, local, soft-updates)
 
 # cat /etc/fstab
 # Device                Mountpoint      FStype  Options         Dump    Pass#
 /dev/ad2s1b             none            swap    sw              0       0
 /dev/ad2s1a             /               ufs     rw              1       1
 /dev/ad2s1e             /home           ufs     rw              2       2
 #/dev/ad2s1h            /nsm1           ufs     rw              2       2
 #/dev/ad3s1e            /nsm2           ufs     rw              2       2
 /dev/concat/nsm         /nsm            ufs     rw              2       2
 /dev/ad2s1f             /tmp            ufs     rw              2       2
 /dev/ad3s1d             /tmp2           ufs     rw              2       2
 /dev/ad2s1d             /usr            ufs     rw              2       2
 /dev/ad2s1g             /var            ufs     rw              2       2
 /dev/acd0               /cdrom          cd9660  ro,noauto       0       0
 
 # bsdlabel /dev/ad3s1
 # /dev/ad3s1:
 8 partitions:
 #        size   offset    fstype   [fsize bsize bps/cpg]
   c: 312496317        0    unused        0     0         # "raw" part,
 don't edit
   d:  1048576        0    4.2BSD     2048 16384     8
   e: 311447741  1048576    4.2BSD     2048 16384 28552
 
 Thank you,
 
 Richard
State-Changed-From-To: open->closed 
State-Changed-By: pjd 
State-Changed-When: wto 11 sie 2009 13:51:58 UTC 
State-Changed-Why:  
As lulf suggested, gconcat has no way to tell if it should use ad5s1d 
or ad5s1 if they present exactly the same data. Gconcat report duplicate, 
but should still work perfectly fine. Either use -h option or ignore 
duplicate report. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120231 

From: Lucas Reddinger <lucas@wingedleopard.net>
To: freebsd-geom@freebsd.org, bug-followup@freebsd.org, taosecurity@gmail.com
Cc: wlw-atlas <wlw-atlas@googlegroups.com>
Subject: Re: kern/120231: [geom] GEOM_CONCAT error adding second drive
Date: Mon, 2 Nov 2009 20:01:06 -0700

 http://www.freebsd.org/cgi/query-pr.cgi?pr=3D120231
 
 I'm having the same problem, but I'm wondering about an alternative
 workaround. Is it possible to remove all partitions, only leaving
 ``c'', so that geom will only attempt to add the whole disk and not
 any other partition as well?
 
 For example, I have:
 
 atlas% sudo bsdlabel /dev/stripe/gs9
 # /dev/stripe/gs9:
 8 partitions:
 #=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size=C2=A0=C2=A0 offset=C2=A0=
 =C2=A0=C2=A0 fstype=C2=A0=C2=A0 [fsize bsize bps/cpg]
 =C2=A0 a: 2500527423=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 16=C2=A0=C2=A0=C2=
 =A0 4.2BSD=C2=A0=C2=A0=C2=A0=C2=A0 2048 16384 28552
 =C2=A0 c: 2500527439=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=
 =A0=C2=A0 unused=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=
 =A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 # "raw"
 part, don't edit
 
 Can I remove the ``a'' partition, only leaving ``c'', so that geom
 only tries to add /dev/stripe/gs9 to /dev/concat/gc9, and not
 /dev/stripe/gs9a as well?
 
 GEOM_CONCAT: Device gc9 created (id=3D3563333761).
 GEOM_CONCAT: Disk stripe/gs9 attached to gc9.
 GEOM_CONCAT: Device gc9 activated.
 GEOM_LABEL: Label for provider stripe/gs9 is ufsid/4aece8d6fd9650d6.
 GEOM_LABEL: Label for provider stripe/gs9a is ufsid/4aeceb954b2a81d6.
 GEOM_CONCAT: Cannot add disk ufsid/4aece8d6fd9650d6 to gc9 (error=3D17).
 
 I appreciate any responses. Thank you!
 
 Lucas Reddinger
>Unformatted:
