From norgaard@charm.daemonsecurity.com  Thu Nov 25 02:25:19 2004
Return-Path: <norgaard@charm.daemonsecurity.com>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 0C84316A4CE
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 25 Nov 2004 02:25:19 +0000 (GMT)
Received: from charm.daemonsecurity.com (FW-182-254.go.retevision.es [62.174.254.182])
	by mx1.FreeBSD.org (Postfix) with ESMTP id BB21F43D60
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 25 Nov 2004 02:25:17 +0000 (GMT)
	(envelope-from norgaard@charm.daemonsecurity.com)
Received: (from root@localhost)
	by charm.daemonsecurity.com (8.13.1/8.13.1) id iAP2PEk0091029;
	Thu, 25 Nov 2004 03:25:14 +0100 (CET)
	(envelope-from norgaard)
Message-Id: <200411250225.iAP2PEk0091029@charm.daemonsecurity.com>
Date: Thu, 25 Nov 2004 03:25:14 +0100 (CET)
From: Erik Norgaard <norgaard@locolomo.org>
Reply-To: Erik Norgaard <norgaard@locolomo.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: PXEBoot problems using md as root device
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         74352
>Category:       kern
>Synopsis:       NFSCLIENT and booting to an mfsroot via TFTP are mutually exclusive
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          suspended
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Nov 25 02:30:23 GMT 2004
>Closed-Date:    
>Last-Modified:  Tue Oct 25 02:02:38 GMT 2005
>Originator:     Erik Norgaard
>Release:        FreeBSD 6.0-CURRENT i386/FreeBSD 5.3-RELEASE
>Organization:
>Environment:
System: VIA C3 Nehemiah CL1000 with dual VIA Rhine III NIC, 256MB RAM, 60GB HDD
dmesg extract (FreeBSD 4.10):
vr0: <VIA VT6105 Rhine III 10/100BaseTX> port 0xd000-0xd0ff mem 0xde000000-0xde0000ff irq 12 at device 15.0 on pci0
vr0: Ethernet address: 00:40:63:d4:89:72
miibus0: <MII bus> on vr0
vr1: <VIA VT6102 Rhine II 10/100BaseTX> port 0xe800-0xe8ff mem 0xde002000-0xde0020ff irq 11 at device 18.0 on pci0
vr1: 00:40:63:d4:89:71
miibus1 <MII bus> on vr1

OS: Tested both FreeBSD 6.0-CURRENT and 5.3-RELEASE with the same custom
configuration file (PXE).

>Description:
I am trying to set up a jumpstart server using dhcp/tftp/ftp and _not_ nfs
to automate installation. There are multiple problems, but they seem related.

In short: The kernel insists on mounting a nfs root device rather than the
provided memory disk.

This happens although:

1) I specify in loader.conf that it should use a memory file system as root
   device (the file is fetched succesfully with tftp).
2) I exclude "option nfs_root" from the kernel configuration file

It does not happen if:

1) I exclude "option nfsclient" from the kernel configuration file. 

BUT: Then the kernel fails to initialise the network device (over which the 
kernel was fetched) even though the network device driver is compiled into the
kernel.

Compiling a kernel with/-out "option BOOTP" does not change the result.

When disabling nfsclient in the kernel, the boot messages does show that the
kernel initialises the second interface vr1, but I cannot manually select it
and I cannot manually configure vr0. There is no problem initializing the
interface when NFSCLIENT is enabled. Also, I have previously had FreeBSD 4.10
installed and both interfaces up working.

The server is running FreeBSD 6.0-CURRENT revision 199506. I have built
the kernel and base system for the client from source, both RELENG_5 (nov 23) 
and -CURRENT (nov 25) as follows:

  make buildworld
  make -DLOADER_TFTP_SUPPORT=YES KERNCONF=PXE buildkernel
  make KERNCONF=PXE DESTDIR=/usr/nclt installkernel
  make DESTDIR=/usr/nclt installworld

More information: I first reported my problems on the freebsd-questions 
mailing list, less structured. All the custom files I have created are 
available with the guide I wrote along the way: www.daemonsecurity.com/pxe/


>How-To-Repeat:

I have described the full process in detail here www.daemonsecurity.com/pxe/. 
Assuming you have the environment setup for pxeboot and no nfsexports:

Update the sources, and build the kernel as described above with the following
options in the kernel configuration file:

   # Filesystems
   options         MD_ROOT         # MD is a potential root device
   options         PSEUDOFS        # Pseudo-filesystem framework

   # Memory pseudo devices
   device          mem             # Memory and kernel memory devices
   device          md              # Memory "disks" needed for root file system

Repeat with/-out:

   options         NFSCLIENT

>Fix:

	


>Release-Note:
>Audit-Trail:

From: Erik Norgaard <norgaard@locolomo.org>
To: FreeBSD-gnats-submit@FreeBSD.org, freebsd-bugs@FreeBSD.org
Cc:  
Subject: Re: kern/74352: PXEBoot problems using md as root device
Date: Thu, 25 Nov 2004 11:15:59 +0100

 I described to problems, they shouldn't be related and I finally
 understood that they are not:
 
 1) NFSCLIENT in the kernel config will override attempts to use a memory
 file as root device
 
 2) Disabling NFSCLIENT made the booting interface unavailable after
 mounting the memory disk device
 
 Problem 1 is still there - however 2 was a problem with my memory file
 system. I created a new bare one just capable of running ifconfig and
 both interfaces show up allthough not configured.
 
 Since 2 is not a bug there is a fix to 1, namely disabling NFSCLIENT.
 
 Erik
 -- 
 Ph: +34.666334818                                  web: www.locolomo.org
 S/MIME Certificate: http://www.locolomo.org/crt/2004071206.crt
 Subject ID:  A9:76:7A:ED:06:95:2B:8D:48:97:CE:F2:3F:42:C8:F2:22:DE:4C:B9
 Fingerprint: 4A:E8:63:38:46:F6:9A:5D:B4:DC:29:41:3F:62:D3:0A:73:25:67:C2
 

From: Erik Norgaard <norgaard@locolomo.org>
To: freebsd-gnats-submit@FreeBSD.org, norgaard@locolomo.org
Cc:  
Subject: Re: kern/74352: PXEBoot problems using md as root device
Date: Sun, 16 Jan 2005 14:55:49 +0100

 Please close this problem-report.
 
 I have come to the conclusion that it was a 42cm error :-) the 
 observations may be added to the documentation though.
 
 Thanks, Erik
 -- 
 Ph: +34.666334818                                  web: www.locolomo.org
 S/MIME Certificate: http://www.locolomo.org/crt/2004071206.crt
 Subject ID:  A9:76:7A:ED:06:95:2B:8D:48:97:CE:F2:3F:42:C8:F2:22:DE:4C:B9
 Fingerprint: 4A:E8:63:38:46:F6:9A:5D:B4:DC:29:41:3F:62:D3:0A:73:25:67:C2
State-Changed-From-To: open->closed 
State-Changed-By: delphij 
State-Changed-When: Sun Jan 16 15:12:33 GMT 2005 
State-Changed-Why:  
Close on submitter's request 


Responsible-Changed-From-To: freebsd-bugs->delphij 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Sun Jan 16 15:12:33 GMT 2005 
Responsible-Changed-Why:  
Take so there's someone to be contacted if this is wrong :-) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=74352 

From: Randy Bias <randyb@kozoru.com>
To: norgaard@locolomo.org, freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/74352: PXEBoot problems using md as root device
Date: Tue, 15 Feb 2005 22:34:59 -0600

 This problem still exists in that NFSCLIENT and booting to an mfsroot 
 via TFTP are mutually exclusive.  I know it's not a common 
 configuration; however, if one were to run say a bunch of embedded 
 diskless systems in a computational cluster it would be useful.  The 
 exact position I find myself in.  The problem is that I need a kernel 
 that allows me to load up mfsroot via TFTP and then mount NFS shares 
 from servers with data to be processed.
 
 I believe this bug should still be open.  There isn't really any reason 
 for NFS/MFS roots to be mutually exclusive.
 
 
 --Randy
 
 
 | Randy L Bias  www.kozoru.com   randyb-at-kozoru-dot-com
 | Director, Product Management                 913.831.2929x26
 | Private:                                    randyb-at-randybias.com
 

From: Erik Norgaard <norgaard@locolomo.org>
To: freebsd-gnats-submit@FreeBSD.org, norgaard@locolomo.org,
	randyb@kozoru.com
Cc:  
Subject: Re: kern/74352: PXEBoot problems using md as root device
Date: Thu, 17 Feb 2005 23:03:12 +0100

 OK, sorry about my own confusion, maybe due to the fact that I initially 
 reported two problems I thought was related but turned out not to be.
 
 The inclusion of NFSCLIENT should not imply NFS_ROOT and should not 
 exclude MD_ROOT. Nor MD_ROOT and NFS_ROOT need be mutually exclusive, 
 since in both cases the root device can be specified on boot - but I 
 think it's ok only to allow only one.
 
 For the setup in which Randy mentions, I don't think that NFSCLIENT in 
 the kernel is strictly required, although without it gets a bit more 
 cumbersome.
 
 You can enable nfs mounts by loading the kernel module nfsclient.ko 
 which can be conveniently placed on the memory disk - for diskless 
 clients, the module is loaded by /etc/rc.d/nfsclient if it is not 
 supported by the kernel - I hope this solvess your problem.
 
 However, if NFSCLIENT implies NFS_ROOT then it must be documented.
 
 In short, there is a workaround, so this is not a critical problem.
 
 Cheers, Erik
 -- 
 Ph: +34.666334818                           web: http://www.locolomo.org
 S/MIME Certificate: http://www.locolomo.org/crt/2004071206.crt
 Subject ID:  A9:76:7A:ED:06:95:2B:8D:48:97:CE:F2:3F:42:C8:F2:22:DE:4C:B9
 Fingerprint: 4A:E8:63:38:46:F6:9A:5D:B4:DC:29:41:3F:62:D3:0A:73:25:67:C2
State-Changed-From-To: closed->open 
State-Changed-By: delphij 
State-Changed-When: Fri Feb 18 02:17:11 GMT 2005 
State-Changed-Why:  
Reopen, so we won't lose the useful information in audit trial. 


Responsible-Changed-From-To: delphij->freebsd-bugs 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Fri Feb 18 02:17:11 GMT 2005 
Responsible-Changed-Why:  
Not before I have time to take a deeper look into this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=74352 
State-Changed-From-To: open->suspended 
State-Changed-By: linimon 
State-Changed-When: Tue Oct 25 01:59:46 GMT 2005 
State-Changed-Why:  
Re-title this with an appropriate Synopsis.  Submitter (and further 
correspondants) note that even though this is not a common setup, it 
"ought to work". 

Mark as suspended because no solution is known and no-one seems to 
be actively working on this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=74352 
>Unformatted:
