From nobody@FreeBSD.org  Sat Feb  1 15:36:37 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id C744BB7E
	for <freebsd-gnats-submit@FreeBSD.org>; Sat,  1 Feb 2014 15:36:37 +0000 (UTC)
Received: from oldred.freebsd.org (oldred.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id B2F9A172B
	for <freebsd-gnats-submit@FreeBSD.org>; Sat,  1 Feb 2014 15:36:37 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id s11Faag1043220
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 1 Feb 2014 15:36:36 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id s11FaadA043213;
	Sat, 1 Feb 2014 15:36:36 GMT
	(envelope-from nobody)
Message-Id: <201402011536.s11FaadA043213@oldred.freebsd.org>
Date: Sat, 1 Feb 2014 15:36:36 GMT
From: John-Paul Bader <contact@smyck.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD 10 Installer creates weird ZFS volume layout for /usr
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         186345
>Category:       misc
>Synopsis:       FreeBSD 10 Installer creates weird ZFS volume layout for /usr
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:
>Keywords:
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat Feb 01 15:40:00 UTC 2014
>Closed-Date:
>Last-Modified:
>Originator:     John-Paul Bader
>Release:        10.0 Release
>Organization:
>Environment:
FreeBSD awesome 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014     root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

>Description:
After installing a fresh FreeBSD 10.0 Release with ZFS root on my server I created a snapshot with the -r option. Then I created a file in /usr/local and wanted to rollback to the previous state but noticed that nothing happened. 

After a bit of investigation I found out that zfs list claims that /usr is a separate volume which gets its own snapshot entries but in reality it is not mounted and instead located in zroot/ROOT/default 

So in order to rollback /usr/local I'd have to rollback zroot/Root/default. 

 zfs get all zroot/usr reports that /usr is not mounted and the canmount option is set to off.

This creates a lot of confusion and is counter intuitive. If you carefully compare the output of zfs list and mount you can see the problem maybe a bit better:

$> mount
zroot/ROOT/default on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
zroot/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot/usr/home on /usr/home (zfs, local, noatime, nfsv4acls)
zroot/usr/ports on /usr/ports (zfs, local, noatime, nosuid, nfsv4acls)
zroot/usr/src on /usr/src (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var on /var (zfs, local, noatime, nfsv4acls)
zroot/var/crash on /var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/mail on /var/mail (zfs, local, nfsv4acls)
zroot/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)

Mount shows no /usr volume but when you look at zfs list you get:

$> zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zroot               2.17G   103G   144K  none
zroot/ROOT           857M   103G   144K  none
zroot/ROOT/default   857M   103G   857M  /
zroot/tmp            192K   103G   192K  /tmp
zroot/usr           1.33G   103G   144K  /usr
zroot/usr/home       184K   103G   184K  /usr/home
zroot/usr/ports      813M   103G   813M  /usr/ports
zroot/usr/src        545M   103G   545M  /usr/src
zroot/var           1.27M   103G   644K  /var
zroot/var/crash      148K   103G   148K  /var/crash
zroot/var/log        212K   103G   212K  /var/log
zroot/var/mail       144K   103G   144K  /var/mail
zroot/var/tmp        152K   103G   152K  /var/tmp

when listing the snapshots again you will get an entry for /usr

zfs list -t snapshot
NAME                      USED  AVAIL  REFER  MOUNTPOINT
zroot@eins                   0      -   144K  -
zroot/ROOT@eins              0      -   144K  -
zroot/ROOT/default@eins     8K      -   857M  -
zroot/tmp@eins               0      -   192K  -
zroot/usr@eins               0      -   144K  -
zroot/usr/home@eins          0      -   184K  -
zroot/usr/ports@eins         0      -   813M  -
zroot/usr/src@eins           0      -   545M  -
zroot/var@eins             96K      -   644K  -
zroot/var/crash@eins         0      -   148K  -
zroot/var/log@eins         84K      -   212K  -
zroot/var/mail@eins          0      -   144K  -
zroot/var/tmp@eins           0      -   152K  -

but in reality /usr and all its sub directories which are not a dedicated volume are under zroot/ROOT/default.

Can you please clean up this mess if there are no real reasons to keep it that way?
>How-To-Repeat:
Install a fresh FreeBSD 10 with ZFS root filesystem with bsdinstall. Make a snapshot, create a file in /usr and try to rollback to the snapshot with zfs rollback zroot/usr@snapshot - /usr will remain the same. Instead do zfs rollback zroot/ROOT/default and the file in /usr will disappear.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted:
