From nobody@FreeBSD.org  Mon Mar 12 03:02:28 2001
Return-Path: <nobody@FreeBSD.org>
Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id 1865937B718
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 12 Mar 2001 03:02:28 -0800 (PST)
	(envelope-from nobody@FreeBSD.org)
Received: (from nobody@localhost)
	by freefall.freebsd.org (8.11.1/8.11.1) id f2CB2Sr94198;
	Mon, 12 Mar 2001 03:02:28 -0800 (PST)
	(envelope-from nobody)
Message-Id: <200103121102.f2CB2Sr94198@freefall.freebsd.org>
Date: Mon, 12 Mar 2001 03:02:28 -0800 (PST)
From: krepel@fokus.gmd.de
To: freebsd-gnats-submit@FreeBSD.org
Subject: [PATCH] mount_mfs is being phased out -> new diskless boot procedure
X-Send-Pr-Version: www-1.0

>Number:         25730
>Category:       conf
>Synopsis:       [PATCH] mount_mfs is being phased out -> new diskless boot procedure
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    bsd
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 12 03:10:01 PST 2001
>Closed-Date:    Sun Aug 12 06:49:46 PDT 2001
>Last-Modified:  Sun Aug 12 06:53:26 PDT 2001
>Originator:     Falco Krepel
>Release:        5.0 current
>Organization:
GMD FOKUS
>Environment:
FreeBSD aries.fokus.gmd.de 5.0-CURRENT FreeBSD 5.0-CURRENT #0: Thu Mar  1 17:29:32 GMT 2001     root@aries.fokus.gmd.de:/usr/obj/usr/src/sys/GENERIC  i386

>Description:
Due to mount_mfs is being phased out in preference for mdconfig, it is
necessary to make a new diskless boot procedure (after every mount_mfs
command the system paused for 15 seconds).

I have replaced the mount_mfs with mdconfig and add a boot switch "swap_enable".
Let me explaine some ideas behind this.

/etc/default/rc.conf:
I have add the "swap_enable" variable to have the knowledge during the boot
process if swap space is avaible or not. Because it is possible to havn't
any swap space. This will be need in the rc.diskless2 file.

/etc/rc.diskless1
Creat a memory disk (md) with malloc. We must use malloc because up to now
no "swapon -a" were made. This is done after rc.diskless1.
Null mount won't needed any more because we can umount the md and 
remount it without any losses.

/etc/rc.diskless2
When swap is enabled make mdconfig with swap is used otherwise make malloc.

Build the var dir from a pre-build var dir. This is easier than build all dirs
and files. With the old procedure many dirs and files are not created at boot
time.

Also the creation of the dev dir is obsolet because of DEVFS working now.
Perhaps it possible to switch between this two solutions depending on
a variable (I don't know if such a variable allready exist).

/etc/defaults/rc.conf
Add the "swap_enable" variable due to the previous explained reasons.
>How-To-Repeat:

>Fix:

aries# diff -u rc.diskless1 /usr/src/etc/rc.diskless1
--- rc.diskless1        Wed Mar  7 17:47:04 2001
+++ /usr/src/etc/rc.diskless1   Sun Oct  8 19:18:24 2000
@@ -98,14 +98,9 @@
 # WARNING! null mounts cannot handle mmap, and since many programs
 # use mmap (such as 'cp'), we have to copy.
 #
-# up to now we don't have make swapon and so we must use malloc
-#
-mdconfig -a -t malloc -s 2048 -u 10
-disklabel -r -w md10 auto 
-newfs /dev/md10c
-mount /dev/md10c /conf/etc
+mount_mfs -s 2048 -T qp120at dummy /conf/etc
 cp -Rp /etc/* /conf/etc
-chkerr $? "MD mount on /conf/etc"
+chkerr $? "MFS mount on /conf/etc"
 
 if [ -d /conf/${bootp_ipa} ] ; then
         cp -Rp /conf/${bootp_ipa}/etc/* /conf/etc
@@ -117,8 +112,7 @@
 
 # Make the new directory available as /etc
 #
-umount /dev/md10c
-mount /dev/md10c /etc
+mount_null /conf/etc /etc
 
 # Tell /etc/rc to run the specified script after
 # it does its mounts but before it does anything


aries# diff -u rc.diskless2 /usr/src/etc/rc.diskless2
--- rc.diskless2        Wed Mar  7 18:03:04 2001
+++ /usr/src/etc/rc.diskless2   Sun Oct  8 19:18:24 2000
@@ -38,29 +38,28 @@
        . /etc/rc.conf
 fi
 
-# If the diskless station work without swap use malloc instead of swap
-#
-case ${swap_enable} in
-[Yy][Ee][Ss])
-       mdconfig -a -t swap -s ${varsize:=65536} -u 11
-       ;;
-*)
-       mdconfig -a -t malloc -s ${varsize:=65536} -u 11
-       ;;
-esac
-
-disklabel -r -w md11 auto
-newfs /dev/md11c
-mount /dev/md11c /mnt
-
-# cp the pre-build /var dir and make available as /var
+mount_mfs -s ${varsize:=65536} -T qp120at dummy /var
+var_dirs="run dev db msgs tmp spool spool/mqueue spool/lpd spool/output \
+       spool/output/lpd"
+for i in ${var_dirs}
+do
+    mkdir /var/${i}
+done
+chmod 755 /var/run
+chmod 755 /var/db
+chmod 755 /var/spool
+chmod 1777 /var/tmp
+chown -R root.daemon /var/spool/output
+chgrp daemon /var/spool/lpd
 #
 # XXX make sure to create one dir for each printer as requested by lpd
 #
-cp -Rp /var/* /mnt
-umount /dev/md11c
-mount /dev/md11c /var
 
 if [ ! -h /tmp -a ! -h /var/tmp ]; then
        mount_null /var/tmp /tmp
 fi
+
+# extract a list of device entries, then copy them to a writable partition
+(cd /; find -x dev | cpio -o -H newc) > /tmp/dev.tmp
+mount_mfs -s 4096 -i 512 -T qp120at dummy /dev
+(cd /; cpio -i -H newc -d < /tmp/dev.tmp)


--- defaults/rc.conf    Wed Mar  7 17:39:08 2001
+++ /usr/src/etc/defaults/rc.conf       Wed Feb 28 22:28:00 2001
@@ -19,7 +19,6 @@
 ###  Important initial Boot-time options  ####################
 ##############################################################
 
-swap_enable="YES"      # Set to NO to disable swapspace
 swapfile="NO"          # Set to name of swapfile if aux swapfile desired.
 apm_enable="NO"                # Set to YES to enable APM BIOS functions (or NO).
 apmd_enable="NO"       # Run apmd to handle APM event from userland.

>Release-Note:
>Audit-Trail:

From: Falco Krepel <krepel@fokus.gmd.de>
To: freebsd-gnats-submit@FreeBSD.org, krepel@fokus.gmd.de
Cc:  
Subject: Re: conf/25730: [PATCH] mount_mfs is being phased out -> new diskless 
 boot procedure
Date: Fri, 30 Mar 2001 18:53:31 +0200

 This is a multi-part message in MIME format.
 --------------65A36429072AABBE6751CEDD
 Content-Type: text/plain; charset=us-ascii
 Content-Transfer-Encoding: 7bit
 
 I have implemented some good ideas from Mike Smith in my
 rc.diskless{1,2} files and make some other changes:
 
 1. Now I use the kernel flag "vm.nswapdev" to determine if swap is
 available.
 
 2. To determine if DEVFS are used I check the kernel flag
 "vfs.devfs.generation" as it done within the /etc/rc file.
 
 3. I write "chk_err" and "mkmd" functions also in rc.diskless2 because
 the rc.diskless2 file is executed in the rc file: "sh rc.diskless2"
 When you change this call to ". rc.diskless2" than the mkmd function
 from rc.diskless2 could replace the mkmd function in rc.diskless1. The
 chk_err funcktion in rc.diskless2 could be deleted.
 
 The following knobs must be added in the /etc/defaults/rc.conf:
 
 diskless_var_size="32m"
 diskless_tmp_enable="NO"
 diskless_tmp_size="32m"
 diskless_dev_size="2m"
 
 Maybe other values are more useful.
 
 -- 
 Falco Krepel			Phone:  +49-(0)30 - 34 63 - 7 276
 GMD-FOKUS			Fax:    +49-(0)30 - 34 63 - 8 276
 Kaiserin-Augusta-Allee 31	e-mail: krepel@fokus.gmd.de
 10589 Berlin			WWW:	http://www.fokus.gmd.de/usr/krepel
 --------------65A36429072AABBE6751CEDD
 Content-Type: text/plain; charset=us-ascii;
  name="rc.diskless1"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline;
  filename="rc.diskless1"
 
 # Copyright (c) 1999  Matt Dillion
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
 # are met:
 # 1. Redistributions of source code must retain the above copyright
 #    notice, this list of conditions and the following disclaimer.
 # 2. Redistributions in binary form must reproduce the above copyright
 #    notice, this list of conditions and the following disclaimer in the
 #    documentation and/or other materials provided with the distribution.
 #
 # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
 # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 # ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 #
 # $FreeBSD: src/etc/rc.diskless1,v 1.6 2000/10/08 19:18:24 obrien Exp $
 #
 
 #
 # /etc/rc.diskless1 - general BOOTP startup
 #
 # BOOTP has mounted / for us.  Assume a read-only mount.  We must then
 # - figure out our IP by querying the interface
 # - fill /conf/etc (writable) with files from /etc, and then update
 #   per-machine files from /conf/*/ where * is the IP of the host,
 #   the IP of the subnet, "default", or nothing.
 # - mount /conf/etc over /etc so we can see the new files.
 #
 # WARNING: i thing you should not change /etc/rc or strange things could
 # happen.
 #
 # The operator is in charge of setting /conf/*/etc/* things as appropriate.
 # Typically rc.conf and fstab need to be changed, but possibly
 # also other files such as inetd.conf etc.
 
 # chkerr:
 #
 # Routine to check for error
 #
 #	checks error code and drops into shell on failure.
 #	if shell exits, terminates script as well as /etc/rc.
 #
 chkerr() {
 	case $1 in
 	0)
 		;;
 	*)
 		echo "$2 failed: dropping into /bin/sh"
 		/bin/sh
 		# RESUME
 		;;
 	esac
 }
 
 # mkmd:
 #
 # Builds an md(4) disk of the size in $1
 # Labels and newfs' it.
 # Mounts it on the destination in $2
 # Returns the name of the created md device in md_device
 # Returns the name of the device containing the filesystem in md_filesystem
 mkmd() {
         md_device=`mdconfig -a -t malloc -s $1`
         chkerr $? "configuring md device"
         disklabel -rw $md_device auto
         chkerr $? "labelling md device"
         md_filesystem=/dev/$md_device"c"
         newfs $md_filesystem 2>&1 >/dev/null
         chkerr $? "making md device filesystem"
         mount $md_filesystem $2
         chkerr $? "mounting md filesystem on $2"
 }
 
 # DEBUGGING
 #
 # set -v
 
 # Figure out our interface and IP.
 #
 bootp_ifc=""
 bootp_ipa=""
 bootp_ipbca=""
 iflist=`ifconfig -l`
 for i in ${iflist} ; do
     set `ifconfig ${i}`
     while [ $# -ge 1 ] ; do
         if [ "${bootp_ifc}" = "" -a "$1" = "inet" ] ; then
             bootp_ifc=${i} ; bootp_ipa=${2} ; shift
         fi
         if [ "${bootp_ipbca}" = "" -a "$1" = "broadcast" ] ; then
             bootp_ipbca=$2; shift
         fi
         shift
     done
     if [ "${bootp_ifc}" != "" ] ; then
         break
     fi
 done
 echo "Interface ${bootp_ifc} IP-Address ${bootp_ipa} Broadcast ${bootp_ipbca}"
  
 # Files in /etc are copied to /conf/etc which is writable. Then
 # per-machine configs from /conf/ip.address/etc are copied onto this
 # directory. First choice is using the client's IP, then the client's
 # broadcast address, then a default configuration.
 # This way we have some flexibility to handle clusters of machines
 # on separate subnets.
 #
 # up to now we don't have make swapon and so we must use malloc
 #
 mkmd 2m /mnt
 cp -Rp /etc/* /mnt
 chkerr $? "copy /etc template"
 
 if [ -d /conf/${bootp_ipa} ] ; then
         cp -Rp /conf/${bootp_ipa}/etc/* /mnt
 elif [ -d /conf/${bootp_ipbca} ] ; then
         cp -Rp /conf/${bootp_ipbca}/etc/* /mnt
 else
         cp -Rp /conf/default/etc/* /mnt
 fi
 
 # Make the new directory available as /etc
 #
 umount /mnt
 mount $md_filesystem /etc
 
 # Tell /etc/rc to run the specified script after
 # it does its mounts but before it does anything
 # else.
 #
 # This script is responsible for setting up the
 # diskless mount environment.  This can be
 # overriden by /conf/ME/rc.conf.local if, for
 # example, you do not want to run the standard
 # system /etc/rc.diskless2
 
 diskless_mount="/etc/rc.diskless2"
 
 --------------65A36429072AABBE6751CEDD
 Content-Type: text/plain; charset=us-ascii;
  name="rc.diskless2"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline;
  filename="rc.diskless2"
 
 # Copyright (c) 1999  Matt Dillon
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
 # are met:
 # 1. Redistributions of source code must retain the above copyright
 #    notice, this list of conditions and the following disclaimer.
 # 2. Redistributions in binary form must reproduce the above copyright
 #    notice, this list of conditions and the following disclaimer in the
 #    documentation and/or other materials provided with the distribution.
 #
 # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
 # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 # ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
 # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 #
 # $FreeBSD: src/etc/rc.diskless2,v 1.7 2000/10/08 19:18:24 obrien Exp $
 #
 
 #
 # rc.diskless2
 #
 
 # If there is a global system configuration file, suck it in.
 #
 if [ -r /etc/defaults/rc.conf ]; then
 	. /etc/defaults/rc.conf
 	source_rc_confs
 elif [ -r /etc/rc.conf ]; then
 	. /etc/rc.conf
 fi
 
 # chkerr:
 #
 # Routine to check for error
 #
 #	checks error code and drops into shell on failure.
 #	if shell exits, terminates script as well as /etc/rc.
 #
 chkerr() {
 	case $1 in
 	0)
 		;;
 	*)
 		echo "$2 failed: dropping into /bin/sh"
 		/bin/sh
 		# RESUME
 		;;
 	esac
 }
 
 # mkmd:
 #
 # Builds an md(4) disk with malloc or swap depending on
 # the diskless client use swap with the size in $1
 # Labels and newfs' it.
 # Mounts it on the destination in $2
 # Returns the name of the created md device in md_device
 # Returns the name of the device containing the filesystem in md_filesystem
 #
 mkmd() {
 	dlv=`/sbin/sysctl -n vm.nswapdev 2> /dev/null`
         if [ ${dlv:=0} != 0 ]; then
 		md_device=`mdconfig -a -t swap -s $1`
 	else        
 		md_device=`mdconfig -a -t malloc -s $1`
 	fi
         chkerr $? "configuring md device"
         disklabel -rw $md_device auto
         chkerr $? "labelling md device"
         md_filesystem=/dev/$md_device"c"
         newfs $md_filesystem 2>&1 >/dev/null
         chkerr $? "making md device filesystem"
         mount $md_filesystem $2
         chkerr $? "mounting md filesystem on $2"
 }
 
 
 # reconstruct /var
 #
 mkmd ${diskless_var_size} /var
 mtree -p /var -eU -f /etc/mtree/BSD.var.dist 2>&1 >/dev/null
 touch /var/log/cron
 touch /var/log/lpd-errs
 touch /var/log/maillog
 touch /var/log/messages
 touch /var/log/ppp.log
 touch /var/log/security
 touch /var/log/slip.log
 
 # Copy the templated /dev into a locally-writable version
 # if wo are not using DEVFS 
 #
 if [ ! sysctl vfs.devfs.generation > /dev/null 2>&1 ]; then
 	mkmd ${diskless_dev_size} /mnt
 	cp -Rp /dev/* /mnt
 	umount /mnt
 	mount $md_filesystem /dev
 fi
 
 # Build a small /tmp
 #
 case ${diskless_tmp_enable} in
 [Yy][Ee][Ss])
 	mkmd ${diskless_tmp_size} /tmp
   	;;
 *)
 	if [ ! -h /tmp -a ! -h /var/tmp ]; then
 		mount_null /var/tmp /tmp
 	fi
 	;;
 esac
 
 --------------65A36429072AABBE6751CEDD--
 
Responsible-Changed-From-To: freebsd-bugs->bsd 
Responsible-Changed-By: dd 
Responsible-Changed-When: Mon Jun 18 18:18:50 PDT 2001 
Responsible-Changed-Why:  
Brian seems to have done some work in this area recently. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=25730 

From: Brian Dean <bsd@bsdhome.com>
To: freebsd-gnats-submit@FreeBSD.org, krepel@fokus.gmd.de
Cc:  
Subject: Re: conf/25730: [PATCH] mount_mfs is being phased out -> new diskless boot procedure
Date: Fri, 29 Jun 2001 23:09:21 -0400

 Most of the requested features have already been added to the
 rc.diskless scripts.  Specifically, mfs has been completely removed
 from -current and a wrapper function has been supplied for mounting
 the memory filesystems.  This same wrapper name is used in -stable,
 albeit, it is defined to use mount_mfs there.  Thus, the only
 difference now between -current and -stable is the definition of the
 'mount_md' shell function.
 
 Additionally:
 
 	* devfs is used automatically if it is available
 
 	* /var is constructed using the /etc/mtree/BSD.var.dist file
           and /var/log/* are initially populated based on the contents
           of /etc/newsyslog.conf instead of hard-coding everything
 
 	* /var and /tmp can be individually sized (using varsize and
           tmpsize) and /tmp can be seperate from /var/tmp now as well
 
 I think the only thing left in the patch that is not already addressed
 is making /var swap-backed if swap_enable is set to 'yes'.  If swap is
 available, then by the time that rc.diskless2 runs, the sysctl
 vm.swap_enabled could be checked instead of supplying a new rc
 variable.  However, one may not want to do this in all cases due to
 swap perhaps being on slow NFS storage, so I'm reluctant to make that
 change.
 
 If the submitter feels this feature is really important, I would
 consider something like 'diskless_var_swapbk=YES' which would be
 checked and cause /var to be set up swap-backed but only if the sysctl
 vm.swap_enable reported true.
 
 But before I spend time on this, please look over the newer code in
 -current and -stable and let us know if it meets your needs.
 
 -Brian
 -- 
 Brian Dean
 bsd@FreeBSD.org
State-Changed-From-To: open->closed 
State-Changed-By: bsd 
State-Changed-When: Sun Aug 12 06:49:46 PDT 2001 
State-Changed-Why:  
No feedback from submitter. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=25730 
>Unformatted:
