From nsayer@quack.kfu.com  Sun Jan  9 02:09:32 2011
Return-Path: <nsayer@quack.kfu.com>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 1BF2A1065696
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  9 Jan 2011 02:09:32 +0000 (UTC)
	(envelope-from nsayer@quack.kfu.com)
Received: from quack.kfu.com (kfu-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:e09::2])
	by mx1.freebsd.org (Postfix) with ESMTP id 9E1E88FC1B
	for <FreeBSD-gnats-submit@freebsd.org>; Sun,  9 Jan 2011 02:09:31 +0000 (UTC)
Received: from quack.kfu.com (localhost [127.0.0.1])
	by quack.kfu.com (8.14.4/8.14.4) with ESMTP id p0929SYt003110
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 8 Jan 2011 18:09:30 -0800 (PST)
	(envelope-from nsayer@quack.kfu.com)
Received: (from nsayer@localhost)
	by quack.kfu.com (8.14.4/8.14.4/Submit) id p0929Ref003109;
	Sat, 8 Jan 2011 18:09:27 -0800 (PST)
	(envelope-from nsayer)
Message-Id: <201101090209.p0929Ref003109@quack.kfu.com>
Date: Sat, 8 Jan 2011 18:09:27 -0800 (PST)
From: Nick Sayer <nsayer@kfu.com>
Reply-To: Nick Sayer <nsayer@kfu.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: boot from zfs kernel.old recovery undocumented/impossible
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         153804
>Category:       kern
>Synopsis:       boot from zfs kernel.old recovery undocumented/impossible
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-fs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          doc-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Jan 09 02:10:04 UTC 2011
>Closed-Date:    Thu May 26 11:56:19 UTC 2011
>Last-Modified:  Thu May 26 16:20:09 UTC 2011
>Originator:     Nick Sayer
>Release:        FreeBSD 8.2-RC1 amd64
>Organization:
Just me
>Environment:
System: FreeBSD quack.kfu.com 8.2-RC1 FreeBSD 8.2-RC1 #5: Sat Jan 8 16:14:48 PST 2011 root@quack.kfu.com:/usr/obj/usr/src/sys/QUACK amd64


	Disk configured for gptzfsboot, root on ZFS
>Description:

After creating an unbootable kernel, the steps necessary to boot
kernel.old are unclear.

The problem is that under normal circumstances, /boot/zfs/zpool.cache is
loaded in some magic way - this is evidenced by breaking into the loader
(by pressing "6") and doing an lsmod.

If you attempt to unload, then manually load kernel and zfs.ko, the
zpool.cache file is not loaded. Attempting to load zpool.cache with the
load command fails, and there is no alternative command to do so.

Attempting to boot the kernel without the zpool.cache loaded results in
an irrecoverable failure to mount root.

>How-To-Repeat:
	Set up a zfsroot system.

	boot.

	hit '6' at the loader menu, then "unload", "load kernel" and "load
	zfs.ko", then "boot".


>Fix:

	Either loading zfs.ko needs to perform whatever magic is
	necessary to fetch zpool.cache, or a command needs to be added
	(like load_geli) to fetch the zpool cache.


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: bcr 
Responsible-Changed-When: Tue May 24 15:31:04 UTC 2011 
Responsible-Changed-Why:  
From the description, it sounds more like an issue for the 
filesystem people.   

http://www.freebsd.org/cgi/query-pr.cgi?pr=153804 

From: Andriy Gapon <avg@FreeBSD.org>
To: bug-followup@FreeBSD.org, nsayer@kfu.com
Cc:  
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Tue, 24 May 2011 21:47:08 +0300

 Try the following options:
 
 1)
 unload
 set kernel=kernel.old
 boot
 
 2)
 unload
 load /boot/kernel.old/kernel
 load -t /boot/zfs/zpool.cache /boot/zfs/zpool.cache
 boot
 
 My expectation is that both should do what you want.
 -- 
 Andriy Gapon

From: Nick Sayer <nsayer@kfu.com>
To: Andriy Gapon <avg@FreeBSD.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Tue, 24 May 2011 14:58:39 -0700

 Option number 2, indeed, does the trick.
 
 This should be added to the documentation for ZFS-on-root installations.
 

From: Pan Tsu <inyaoo@gmail.com>
To: Andriy Gapon <avg@FreeBSD.org>
Cc: bug-followup@FreeBSD.org, Nick Sayer <nsayer@kfu.com>
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Wed, 25 May 2011 12:52:38 +0400

 Andriy Gapon <avg@FreeBSD.org> writes:
 
 > Try the following options:
 > 
 > 1)
 > unload
 > set kernel=kernel.old
 > boot
 > 
 > 2)
 > unload
 > load /boot/kernel.old/kernel
 > load -t /boot/zfs/zpool.cache /boot/zfs/zpool.cache
 
 zpool_cache_type can be anything. loader(8) unlike grub2 doesn't enforce
 type to be "/boot/zfs/zpool.cache".
 
 > boot
 
 Whatever happened to just typing following?
 
   boot kernel.old
 
 It assumes zpool.cache was already preloaded by failed attempt to boot
 default kernel. No need to `unload' the kernel, `boot' replaces it.
 
 > My expectation is that both should do what you want.

From: Nick Sayer <nsayer@kfu.com>
To: Pan Tsu <inyaoo@gmail.com>
Cc: Andriy Gapon <avg@FreeBSD.org>, bug-followup@FreeBSD.org
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Wed, 25 May 2011 07:21:40 -0700

 On May 25, 2011, at 1:52 AM, Pan Tsu wrote:
 
 > Andriy Gapon <avg@FreeBSD.org> writes:
 >=20
 >> Try the following options:
 >>=20
 >> 1)
 >> unload
 >> set kernel=3Dkernel.old
 >> boot
 >>=20
 >> 2)
 >> unload
 >> load /boot/kernel.old/kernel
 >> load -t /boot/zfs/zpool.cache /boot/zfs/zpool.cache
 >=20
 > zpool_cache_type can be anything. loader(8) unlike grub2 doesn't =
 enforce
 > type to be "/boot/zfs/zpool.cache".
 >=20
 >> boot
 >=20
 > Whatever happened to just typing following?
 >=20
 >  boot kernel.old
 >=20
 > It assumes zpool.cache was already preloaded by failed attempt to boot
 > default kernel. No need to `unload' the kernel, `boot' replaces it.
 
 I didn't try that. At the time I didn't think it worked that way. Back =
 when I was in full on panic mode, I think option number 1 was what =
 finally worked, but I am happier with option 2, as it offers the most =
 control. It just needs to be better documented.
 
 >=20
 >> My expectation is that both should do what you want.
 
State-Changed-From-To: open->closed 
State-Changed-By: avg 
State-Changed-When: Thu May 26 11:55:22 UTC 2011 
State-Changed-Why:  
No change needed. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=153804 

From: Andriy Gapon <avg@FreeBSD.org>
To: Pan Tsu <inyaoo@gmail.com>, bug-followup@FreeBSD.org,
        Nick Sayer <nsayer@kfu.com>
Cc:  
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Thu, 26 May 2011 14:54:53 +0300

 on 25/05/2011 11:52 Pan Tsu said the following:
 > Andriy Gapon <avg@FreeBSD.org> writes:
 > 
 >> Try the following options:
 >>
 >> 1)
 >> unload
 >> set kernel=kernel.old
 >> boot
 >>
 >> 2)
 >> unload
 >> load /boot/kernel.old/kernel
 >> load -t /boot/zfs/zpool.cache /boot/zfs/zpool.cache
 > 
 > zpool_cache_type can be anything. loader(8) unlike grub2 doesn't enforce
 > type to be "/boot/zfs/zpool.cache".
 
 Yes, one can specify load -t anything /boot/zfs/zpool.cache and it will work.
 I was just on the safe side following what's done in /boot/defaults/loader.conf.
 
 >> boot
 > 
 > Whatever happened to just typing following?
 > 
 >   boot kernel.old
 > 
 > It assumes zpool.cache was already preloaded by failed attempt to boot
 > default kernel. No need to `unload' the kernel, `boot' replaces it.
 > 
 
 Yes, I think that this command is a more convenient form of doing #1 above.
 
 All in all, I qualify this as a pilot error.
 Supported convenience methods do the right thing automatically.  In the completely
 manual method a user should know what he is doing.  Perhaps it's worth mentioning
 somewhere that zpool.cache must be loaded, but I think that it's a common knowledge.
 
 -- 
 Andriy Gapon

From: Nick Sayer <nsayer@kfu.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Thu, 26 May 2011 07:31:13 -0700

 Before this gets closed, is someone going to maybe add this to some =
 documentation somewhere? This ought to go in the handbook at the very =
 least.
 

From: Martin Simmons <martin@lispworks.com>
To: bug-followup@FreeBSD.org, nsayer@kfu.com
Cc:  
Subject: Re: kern/153804: boot from zfs kernel.old recovery undocumented/impossible
Date: Thu, 26 May 2011 17:01:10 +0100

 >>>>> On Thu, 26 May 2011 18:20:14 +0300, Andriy Gapon said:
 > 
 > on 26/05/2011 17:40 Nick Sayer said the following:
 > >  
 > >  Before this gets closed, is someone going to maybe add this to some =
 > >  documentation somewhere? This ought to go in the handbook at the very =
 > >  least.
 > 
 > Which part exactly?
 > IMO, loader(8) should be sufficient.
 
 I think it should be
 
 http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-trouble.html
 
 where it currently tells you to do unload and boot with arguments that don't
 work for zfs...
 
 __Martin
>Unformatted:
