From nobody@FreeBSD.org  Mon Nov  7 00:09:34 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 78B3810656A5
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  7 Nov 2011 00:09:34 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 4A5838FC24
	for <freebsd-gnats-submit@FreeBSD.org>; Mon,  7 Nov 2011 00:09:34 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id pA709Y3Q075273
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 7 Nov 2011 00:09:34 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id pA709XKM075272;
	Mon, 7 Nov 2011 00:09:33 GMT
	(envelope-from nobody)
Message-Id: <201111070009.pA709XKM075272@red.freebsd.org>
Date: Mon, 7 Nov 2011 00:09:33 GMT
From: Jeff Lawson <jeff@bovine.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD hides gpt labels after mounting ZFS partitions
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         162342
>Category:       kern
>Synopsis:       FreeBSD hides gpt labels after mounting ZFS partitions
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Nov 07 00:10:10 UTC 2011
>Closed-Date:    
>Last-Modified:  Tue Nov 15 18:30:05 UTC 2011
>Originator:     Jeff Lawson
>Release:        8.2-RELEASE-p2
>Organization:
>Environment:
FreeBSD mfsbsd 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #3 r222557M: Thu Jun 16 23:58:02 CEST 2011     root@neo.vx.sk:/usr/obj/releng_8_2/sys/GENERIC  amd64

>Description:
If you use "gpart" to create a GPT disk with partitions that have gpt labels defined, then those labels are normally visible in /dev/gpt/

However, once any of those partitions are used by a ZFS pool that is imported, the label disappears from /dev/gpt/

After you stop using that ZFS pool with "zpool export", the missing labels do not reappear in /dev/gpt/ until you reboot.  The GUID entries in /dev/gptid/ are fine and do not disappear after importing the ZFS pool.


This forum thread from 5 months ago is about another user that has experienced similar frustration by this problem. (He ended up with a workaround of using the older style "glabel" which creates labels under /dev/label instead of relying on the GPT-specific labels):  http://forums.freebsd.org/showthread.php?t=24695

>How-To-Repeat:
The tutorial at this wiki page describes using "gpart add -l" to define labels and create an environment that replicates this problem:
http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror

In particular, this command will create a GPT partition with a label:
Fixit# gpart add -s 60G -t freebsd-zfs -l disk0 ad0


Here is some actual output from my own system:

mfsbsd# ls -l /dev/gpt
total 0
crw-r-----  1 root  operator    0,  87 Nov  6 17:50 data0
crw-r-----  1 root  operator    0,  94 Nov  6 17:50 data1
crw-r-----  1 root  operator    0,  89 Nov  6 17:50 swap0
crw-r-----  1 root  operator    0,  92 Nov  6 17:50 swap1

(labels are present, as they should be)


mfsbsd# mkdir /zroot
mfsbsd# zpool import -R /zroot zroot
mfsbsd# ls -l /dev/gpt
total 0
crw-r-----  1 root  operator    0,  89 Nov  6 17:50 swap0
crw-r-----  1 root  operator    0,  92 Nov  6 17:50 swap1

(labels have disappeared after importing the ZFS pool)


mfsbsd# uname -a
FreeBSD mfsbsd 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #3 r222557M: Thu Jun 16 23:58:02 CEST 2011     

root@neo.vx.sk:/usr/obj/releng_8_2/sys/GENERIC  amd64
mfsbsd#


mfsbsd# gpart list | egrep 'Name|label'
1. Name: ad2p1
   label: (null)
2. Name: ad2p2
   label: data0               <----
3. Name: ad2p3
   label: swap0
1. Name: ad2
1. Name: ad3p1
   label: (null)
2. Name: ad3p2
   label: data1               <----
3. Name: ad3p3
   label: (null)
1. Name: ad3

>Fix:


>Release-Note:
>Audit-Trail:

From: Garrett Wollman <wollman@hergotha.csail.mit.edu>
To: jeff@bovine.net
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: kern/162342: FreeBSD hides gpt labels after mounting ZFS partitions
Date: Sun, 6 Nov 2011 23:17:04 -0500 (EST)

 jeff@bovine.net submitted the following bug report:
 
 >If you use "gpart" to create a GPT disk with partitions that have gpt
 >labels defined, then those labels are normally visible in /dev/gpt/
 >
 >However, once any of those partitions are used by a ZFS pool that is
 >imported, the label disappears from /dev/gpt/
 
 Not in my experience.  They only disappear if ZFS is opening the
 underlying partition devices rather than the label devices -- i.e., in
 the usual case, you did a "zpool create foo /dev/ada0p2 /dev/ada1p2"
 rather than "zpool create foo /dev/gpt/data0 /dev/gpt/data1".  I
 suppose that if you created the filesystem on another device and let
 "zfs import" find it by groveling around through all the GEOM
 providers on the system, this might happen.
 
 What does "strings /boot/zfs/zpool.cache | fgrep /dev" say?
 
 -GAWollman
 

From: Jeff Lawson <jeff@bovine.net>
To: freebsd-gnats-submit@freebsd.org
Cc:  
Subject: Re: kern/162342: FreeBSD hides gpt labels after mounting ZFS partitions
Date: Sun, 6 Nov 2011 22:43:34 -0600

 --14dae9340981c92a0604b11db384
 Content-Type: text/plain; charset=UTF-8
 
 My /etc/zfs/zpool.cache indeed references only the /dev/gptid/ GUIDs for
 the drives.  In my case, I added the drive to the pool with its GPT label
 using a /dev/gpt/ specifier.  I verified that "zpool status" showed the new
 drive being resilvered and that it used a gpt label.  After the resilver
 completed, I rebooted, had to use "zpool import", and observed that "zpool
 status" now used a gptid for both drives.
 
 Regardless of whether ZFS is using gptid to access a disk, I don't see why
 the label should disappear.  It should continue to be available as an alias
 if the user wants to reference it.
 
 --14dae9340981c92a0604b11db384
 Content-Type: text/html; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable
 
 My /etc/zfs/zpool.cache indeed references only the /dev/gptid/ GUIDs for th=
 e drives.=C2=A0 In my case, I added the drive to the pool with its GPT labe=
 l using a /dev/gpt/ specifier.=C2=A0 I verified that &quot;zpool status&quo=
 t; showed the new drive being resilvered and that it used a gpt label.=C2=
 =A0 After the resilver completed, I rebooted, had to use &quot;zpool import=
 &quot;, and observed that &quot;zpool status&quot; now used a gptid for bot=
 h drives.<br>
 
 <br>Regardless of whether ZFS is using gptid to access a disk, I don&#39;t =
 see why the label should disappear.=C2=A0 It should continue to be availabl=
 e as an alias if the user wants to reference it.<br>
 
 --14dae9340981c92a0604b11db384--

From: Garrett Wollman <wollman@hergotha.csail.mit.edu>
To: jeff@bovine.net
Cc: freebsd-gnats-submit@freebsd.org
Subject: Re: kern/162342: FreeBSD hides gpt labels after mounting ZFS
	partitions
Date: Mon, 7 Nov 2011 00:20:30 -0500 (EST)

 In article <201111070510.pA75A7x2010669@freefall.freebsd.org> you write:
 
 > Regardless of whether ZFS is using gptid to access a disk, I don't see why
 > the label should disappear.  It should continue to be available as an alias
 > if the user wants to reference it.
 
 The label device node is just another client to the underlying disk
 partition provider.  As soon as you open the disk for write via one
 path, all of the other paths that reference the same area of the disk
 are automatically destroyed (since you wouldn't be allowed to open
 them anyway).
 
 -GAWollman
 

From: Jeff Lawson <jeff@bovine.net>
To: freebsd-gnats-submit@freebsd.org
Cc:  
Subject: Re: kern/162342: FreeBSD hides gpt labels after mounting ZFS partitions
Date: Sun, 6 Nov 2011 23:40:52 -0600

 --14dae9341151b505f904b11e806e
 Content-Type: text/plain; charset=UTF-8
 
 Should the GPT labels reappear after using zpool export (or even zpool
 offline)?  That would at least provide a repeatable experience.
 
 From that forum thread I referenced, the user seemed to indicate no such
 problem with disappearing glabel-style labels.  Would you expect that
 difference?
 
 --14dae9341151b505f904b11e806e
 Content-Type: text/html; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable
 
 Should the GPT labels reappear after using zpool export (or even zpool offl=
 ine)?=C2=A0 That would at least provide a repeatable experience.<br><br>Fro=
 m that forum thread I referenced, the user seemed to indicate no such probl=
 em with disappearing glabel-style labels.=C2=A0 Would you expect that diffe=
 rence?<br>
 
 <br><br>
 
 --14dae9341151b505f904b11e806e--

From: David Evans <dave.evans55@googlemail.com>
To: bug-followup@FreeBSD.org, jeff@bovine.net
Cc:  
Subject: Re: kern/162342: FreeBSD hides gpt labels after mounting ZFS partitions
Date: Tue, 15 Nov 2011 12:05:21 +0000

 I can't get gpt labels to appear in /dev/gpt on my first disk, but
 they work on the second and subsequent disks.
 
 10.0-CURRENT amd64 as of a few days ago. I've noticed this on other
 versions as well.  I don't even have ZFS installed.
 
 gpart list shows the labels.
 
 kern.geom.label.gpt.enable sysctl is set to 1

From: David Evans <dave.evans55@googlemail.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/162342: FreeBSD hides gpt labels after mounting ZFS partitions
Date: Tue, 15 Nov 2011 18:28:06 +0000

 It seems to be quite legitimate for the entries in /dev/gpt
 to disappear when a filesystem is mounted.  I set
 kern.geom.label.debug to 255 at the boot loader prompt
 to see what was happening and they were indeed deleted.
 
 The real bug is why are the entries for filesystems on
 /dev/ada1, the second drive, not being deleted when
 the file system is mounted?  The same applies to the
 entries in /dev/ufsid and /dev/gptid.  There is some
 inconsistency here, which has had me a bit confused.
 
 Tested on 10.0-CURRENT and 9.0-RC2
 The 9.0-RC2 system has several drives with gpt
 labels, and they all appear except for the filesystem
 on /dev/ada0.
 
>Unformatted:
