From mexas@bristol.ac.uk  Mon Jul 13 12:02:13 2009
Return-Path: <mexas@bristol.ac.uk>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CCAEE106566C
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 13 Jul 2009 12:02:13 +0000 (UTC)
	(envelope-from mexas@bristol.ac.uk)
Received: from dirg.bris.ac.uk (dirg.bris.ac.uk [137.222.10.102])
	by mx1.freebsd.org (Postfix) with ESMTP id 7CA058FC24
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 13 Jul 2009 12:02:13 +0000 (UTC)
	(envelope-from mexas@bristol.ac.uk)
Received: from isis.bris.ac.uk ([137.222.10.63])
	by dirg.bris.ac.uk with esmtp (Exim 4.69)
	(envelope-from <mexas@bristol.ac.uk>)
	id 1MQJyX-0005Li-Nk
	for FreeBSD-gnats-submit@freebsd.org; Mon, 13 Jul 2009 12:45:47 +0100
Received: from mech-cluster238.men.bris.ac.uk ([137.222.187.238])
	by isis.bris.ac.uk with esmtp (Exim 4.67)
	(envelope-from <mexas@bristol.ac.uk>)
	id 1MQJxG-0004ra-Tj
	for FreeBSD-gnats-submit@freebsd.org; Mon, 13 Jul 2009 12:45:41 +0100
Received: from mech-cluster238.men.bris.ac.uk (localhost.men.bris.ac.uk [127.0.0.1])
	by mech-cluster238.men.bris.ac.uk (8.14.3/8.14.3) with ESMTP id n6DBiM5a042672
	for <FreeBSD-gnats-submit@freebsd.org>; Mon, 13 Jul 2009 12:44:22 +0100 (BST)
	(envelope-from mexas@mech-cluster238.men.bris.ac.uk)
Received: (from mexas@localhost)
	by mech-cluster238.men.bris.ac.uk (8.14.3/8.14.3/Submit) id n6DBiMjW042671;
	Mon, 13 Jul 2009 12:44:22 +0100 (BST)
	(envelope-from mexas)
Message-Id: <200907131144.n6DBiMjW042671@mech-cluster238.men.bris.ac.uk>
Date: Mon, 13 Jul 2009 12:44:22 +0100 (BST)
From: Anton Shterenlikht <mexas@bristol.ac.uk>
Reply-To: Anton Shterenlikht <mexas@bristol.ac.uk>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: draft new section on gmirror per partition
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         136712
>Category:       docs
>Synopsis:       [handbook] [patch] draft new section on gmirror per partition
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-doc
>State:          closed
>Quarter:        
>Keywords:       handbook
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jul 13 12:10:01 UTC 2009
>Closed-Date:    Wed Mar 05 03:23:24 UTC 2014
>Last-Modified:  Wed Mar 05 03:23:24 UTC 2014
>Originator:     Anton Shterenlikht
>Release:        FreeBSD 8.0-BETA1 ia64
>Organization:
University of Bristol, UK
>Environment:
System: FreeBSD 8.0-BETA1 #0: Sun Jul 12 15:15:30 BST 2009     mexas@tzav.vms.org:/usr/obj/usr/src/sys/TZAV  ia64

	
>Description:
	section 19.4 RAID1 - Mirroring is incorrect for ia64 architecture,
	or more generally, for GPT partitioning scheme. One mirroring
	individual partitions would work for GPT, mirroring a whole disk
	wouldn't. So I added a section "Mirroring individual partitions",
	and a subsection under Troubleshooting.
>How-To-Repeat:
	
>Fix:
	section 19.4 with the proposed changes is at
	http://eis.bris.ac.uk/~mexas/geom-mirror.html

>Release-Note:
>Audit-Trail:

From: Anton Shterenlikht <mexas@bristol.ac.uk>
To: bug-followup@FreeBSD.org
Cc:  
Subject: RE: docs/136712: [handbook] [patch] draft new section on gmirror
 per partition
Date: Fri, 14 Aug 2009 11:58:53 +0100

 below is the proposed patch
 
 ############
 
 
 --- /usr/build/doc/en_US.ISO8859-1/books/handbook/geom/chapter.sgml	2009-05-06 15:12:13.000000000 +0100
 +++ ./chapter.sgml	2009-08-14 11:31:05.000000000 +0100
 @@ -220,10 +220,10 @@
  
      <screen>&prompt.root; <userinput>echo 'geom_stripe_load="YES"' &gt;&gt; /boot/loader.conf</userinput></screen>
    </sect1>
 -
    <sect1 id="GEOM-mirror">
 -    <title>RAID1 - Mirroring</title>
  
 +    <title>RAID1 - Mirroring</title>
 +	
      <indexterm>
        <primary>GEOM</primary>
      </indexterm>
 @@ -375,6 +375,295 @@
      </sect2>
  
      <sect2>
 +
 +    <sect2info>
 +      <authorgroup>
 +        <author>
 +          <firstname>Anton</firstname>
 +          <surname>Shterenlikht</surname>
 +          <contrib>With contributions from </contrib>
 +        </author>
 +      </authorgroup>
 +    </sect2info>
 +
 +
 +      <title>Mirroring individual partitions</title>
 +
 +	  <note>
 +	    <para>in some partitioning schemes mirroring a whole disk
 +		is not possible at all. The most notable
 +		example is <acronym role="GUID Partition Table">GPT</acronym>,
 +		used in ia64 architecture. Secondary <acronym>GPT</acronym>
 +		header is stored in the last sector on disk. Since
 +		<command>gmirror</command> writes its data in the last sector,
 +		it destroys secondary <acronym>GPT</acronym>.
 +		A very good introduction to
 +	<ulink url="http://en.wikipedia.org/wiki/GUID_Partition_Table">GPT
 +		(GUID Partition Table)</ulink> can be found on Wikipedia.</para>
 +	</note>
 +
 +	<para>Mirroring individual partitions is a more
 +	  flexible way of achieving <acronym>RAID1</acronym> compared to
 +	  mirroring a whole disk. Below is a step by step guide.</para>
 +
 +	<orderedlist>
 +	  <listitem>
 +	    <para>Check partition of the boot disk using &man.gpart.8;:</para>
 +<screen>&prompt.root; <userinput>gpart show</userinput>
 +=>       34  143374671  da0  GPT  (68G)
 +         34     819200    1  efi  (400M)
 +     819234    1048576    2  freebsd-ufs  (512M)
 +    1867810    4194304    3  freebsd-swap  (2.0G)
 +    6062114    2097152    4  freebsd-ufs  (1.0G)
 +    8159266    2097152    5  freebsd-ufs  (1.0G)
 +   10256418  133118287    6  freebsd-ufs  (63G)</screen>
 +	    <note>
 +	      <para>this example is taken from ia64 architecture,
 +		which includes EFI boot partition. Depending on the
 +		architecture of your system you might or might not have
 +		this partition present.</para>
 +	    </note>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Partition a spare disk, <devicename>da1</devicename>,
 +		exactly as the boot disk. First create a partitioning
 +		scheme on <devicename>da1</devicename>, identical to
 +		that on <devicename>da0</devicename>, in this example
 +		this is <acronym>GPT</acronym>:</para>
 +<screen>&prompt.root; <userinput>gpart create -s gpt da1</userinput></screen>
 +	    <para>Add partitions exactly as on
 +		<devicename>da0</devicename>:</para>
 +<screen>&prompt.root; <userinput>gpart add -b 34 -s 819200 -t freebsd-efi da1</userinput>
 +&prompt.root; <userinput>gpart add -b 819234 -s 1048576 -t freebsd-ufs da1</userinput>
 +&prompt.root; <userinput>gpart add -b 1867810 -s 4194304 -t freebsd-swap da1</userinput>
 +&prompt.root; <userinput>gpart add -b 6062114 -s 2097152 -t freebsd-ufs da1</userinput>
 +&prompt.root; <userinput>gpart add -b 8159266 -s 2097152 -t freebsd-ufs da1</userinput>
 +&prompt.root; <userinput>gpart add -b 10256418 -s 133118287 -t freebsd-ufs da1</userinput></screen>
 +	    <note>
 +		<para><literal>-b</literal> sets the starting block of
 +		  a new partition, <literal>-s</literal> specifies partition
 +		  size, in blocks, and <literal>-t</literal> is partition
 +		  type. On ia64 systems the boot partition is of EFI type,
 +		  hence the first partition added with <command>gpart</command>
 +		  is <literal>freebsd-efi</literal>. Likewise,
 +		  <literal>freebsd-swap</literal> is used to add a
 +		  <literal>swap</literal> partition.
 +		</para>
 +	    </note>
 +	    <tip>
 +		<para>do <literal>gpart show</literal> before each
 +		  <literal>gpart add</literal>. This will tell you the
 +		  starting block.</para> 
 +	    </tip>
 +	    <para>When you are done both disks should be partitioned
 +		identically:</para>
 +<screen>&prompt.root; <userinput>gpart show</userinput>
 +=>       34  143374671  da0  GPT  (68G)
 +         34     819200    1  efi  (400M)
 +     819234    1048576    2  freebsd-ufs  (512M)
 +    1867810    4194304    3  freebsd-swap  (2.0G)
 +    6062114    2097152    4  freebsd-ufs  (1.0G)
 +    8159266    2097152    5  freebsd-ufs  (1.0G)
 +   10256418  133118287    6  freebsd-ufs  (63G)
 +
 +=>       34  143374671  da1  GPT  (68G)
 +         34     819200    1  efi  (400M)
 +     819234    1048576    2  freebsd-ufs  (512M)
 +    1867810    4194304    3  freebsd-swap  (2.0G)
 +    6062114    2097152    4  freebsd-ufs  (1.0G)
 +    8159266    2097152    5  freebsd-ufs  (1.0G)
 +   10256418  133118287    6  freebsd-ufs  (63G)</screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>load gmirror kernel module:</para>
 +<screen>&prompt.root; <userinput>gmirror load</userinput></screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Create mirror for EFI partition.<para>
 +	    <note>
 +	      <para>If you do not have an EFI partition, skip this step.</para>
 +	    </note>
 +	    <para>Unmount <filename>/efi</filename> because GEOM
 +		manipulations can be performed only on unmounted,
 +		not in use, partition:</para>
 +<screen>&prompt.root; <userinput>umount /efi</userinput></screen>
 +	    <para>create EFI mirror on the <emphasis>boot</emphasis> disk,
 +		<devicename>da0</devicename>, in our example:
 +<screen>&prompt.root; <userinput>gmirror label -vb round-robin efi da0p1</userinput></screen>
 +	    <para>This would create <filename>/dev/mirror/efi</filename>
 +		device.<para>
 +	    <para>Add EFI partition of the spare disk,
 +		<devicename>da1</devicename> to the mirror:</para>
 +<screen>&prompt.root; <userinput>gmirror insert efi da1p1</userinput></screen>
 +	    <para>The EFI partition in this example is only 400MB, so it
 +		rebuilds quickly:</para>
 +<screen>&prompt.root; <userinput>gmirror status</userinput>
 +      Name    Status  Components
 +mirror/efi  COMPLETE  da0p1
 +                      da1p1</screen>
 +	    <para>mount EFI mirror and check:</para>
 +<screen>&prompt.root; <userinput>mount -t msdosfs /dev/mirror/efi /efi</userinput>
 +&prompt.root; <userinput>df</userinput>
 +Filesystem       512-blocks    Used     Avail Capacity  Mounted on
 +...
 +/dev/mirror/efi      819008  141728    677280    17%    /efi</screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Create mirror for root (<filename>/</filename>) partition.
 +		This involves extra steps since <filename>/</filename>
 +		cannot be unmounted.</para>
 +	    <para>Create mirror on the spare disk,
 +		<devicename>da1</devicename>:</para>
 +<screen>&prompt.root; <userinput>gmirror label -vb round-robin root da1p2</userinput></screen>
 +	    <para>Create ufs filesystem on this mirror:</para>
 +<screen>&prompt.root; <userinput>newfs /dev/mirror/root</userinput></screen>
 +	    <para>Mount root mirror temporarily, say under
 +		<filename>/mnt</filename>:</para>
 +<screen>&prompt.root; <userinput>mount /dev/mirror/root /mnt</userinput></screen>
 +	    <para>Copy <filename>/</filename> onto <filename>/mnt</filename>
 +		(which is the root mirror,
 +		<filename>/dev/mirror/root</filename>), using a
 +		combination of &man.dump.8; and &man.restore.8;:</para>
 +<screen>&prompt.root; <userinput>cd /mnt</userinput>
 +&prompt.root; <userinput>dump 0aLf - / | restore rf -</userinput></screen>
 +	    <warning>
 +	      <para>&man.dump.8; is the only safe way to copy root
 +		partition. Any other copying tool is not guaranteed
 +		to do it right.</para>
 +	    </warning>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Update <filename>fstab</filename> on the
 +		<emphasis>mirror</emphasis>. Edit
 +		<filename>/mnt/etc/fstab</filename> and change
 +		<devicename>da0p1</devicename> into
 +		<devicename>mirror/efi</devicename> and
 +		<devicename>da0p2</devicename> into
 +		<devicename>mirror/root</devicename>:</para>
 +<screen>&prompt.root; <userinput>cat /mnt/etc/fstab</userinput>
 +# Device                Mountpoint      FStype  Options         Dump    Pass#
 +/dev/da0p3              none            swap    sw              0       0
 +/dev/mirror/root        /               ufs     rw              1       1
 +/dev/mirror/efi         /efi            msdosfs rw              0       0
 +/dev/da0p5              /tmp            ufs     rw              2       2
 +/dev/da0p6              /usr            ufs     rw              2       2
 +/dev/da0p4              /var            ufs     rw              2       2
 +/dev/acd0               /cdrom          cd9660  ro,noauto       0       0</screen>
 +	   </listitem>
 +
 +	  <listitem>
 +	    <para>Enable loading of the <filename>geom_mirror.ko</filename>
 +		kernel module during system initialization. For this do
 +		either:</para>
 +<screen>&prompt.root; <userinput>echo 'geom_mirror_load="YES"' &gt;&gt; /boot/loader.conf</userinput></screen>
 +	    <para>or add
 +	      <programlisting>options         GEOM_MIRROR</programlisting>
 +		to your kernel configuration file.</para>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Change the root device in
 +		<filename>/boot/loader.conf</filename>. The root
 +		device is specified with <literal>vfs.root.mountfrom</literal>
 +		option. It should point to the root mirror. For example, change
 +<programlisting>vfs.root.mountfrom="ufs:/dev/da0p2"</programlisting>
 +		into
 +<programlisting>vfs.root.mountfrom="ufs:/dev/mirror/root"</programlisting>
 +	    </para>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Reboot into <emphasis>single user</emphasis> mode.</para>
 +		<tip>
 +		  <para>On ia64 type <literal>boot -s</literal> on the
 +		boot prompt.</para>
 +		</tip>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>At boot you should see <literal>gmirror</literal> loaded,
 +		and then these lines:</para>
 +<screen>GEOM_MIRROR: Device mirror/efi launched (2/2).
 +GEOM_MIRROR: Device mirror/root launched (1/1).
 +Trying to mount root from ufs:/dev/mirror/root</screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Now that <devicename>da0p2</devicename> is not
 +		mounted, it can be inserted into root mirror:
 +<screen>&prompt.root; <userinput>gmirror insert root da0p2</userinput></screen>
 +		and after mirror rebuild is complete you should see:
 +<screen>&prompt.root; <userinput>gmirror status</userinput>
 +       Name    Status  Components
 + mirror/efi  COMPLETE  da0p1
 +                       da1p1
 +mirror/root  COMPLETE  da1p2
 +                       da0p2</screen>
 +	    <para>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Create mirrors for all other partitions of
 +		<devicename>da0</devicename>, which are now not mounted:</para>
 +<screen>&prompt.root; <userinput>gmirror label -vb round-robin swap da0p3</userinput>
 +&prompt.root; <userinput>gmirror label -vb round-robin var da0p4</userinput>
 +&prompt.root; <userinput>gmirror label -vb round-robin tmp da0p5</userinput>
 +&prompt.root; <userinput>gmirror label -vb round-robin usr da0p6</userinput></screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Edit <filename>/etc/fstab</filename> and change each remaining
 +		<devicename>da0</devicename> partition into its mirror:</para>
 +<screen>&prompt.root; <userinput>cat /etc/fstab</userinput>
 +# Device                Mountpoint      FStype  Options         Dump    Pass#
 +/dev/mirror/swap        none            swap    sw              0       0
 +/dev/mirror/root        /               ufs     rw              1       1
 +/dev/mirror/efi         /efi            msdosfs rw              0       0
 +/dev/mirror/tmp         /tmp            ufs     rw              2       2
 +/dev/mirror/usr         /usr            ufs     rw              2       2
 +/dev/mirror/var         /var            ufs     rw              2       2
 +/dev/acd0               /cdrom          cd9660  ro,noauto       0       0</screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Reboot.</para>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Add remaining <devicename>da1</devicename> partitions
 +		to mirrors:</para>
 +<screen>&prompt.root; <userinput>gmirror insert swap da1p3</userinput>
 +&prompt.root; <userinput>gmirror insert var da1p4</userinput>
 +&prompt.root; <userinput>gmirror insert tmp da1p5</userinput>
 +&prompt.root; <userinput>gmirror insert usr da1p6</userinput></screen>
 +	  </listitem>
 +
 +	  <listitem>
 +	    <para>Done!:</para>
 +<screen>&prompt.root; <userinput>gmirror status</userinput>
 +       Name    Status  Components
 + mirror/efi  COMPLETE  da0p1
 +                       da1p1
 +mirror/root  COMPLETE  da0p2
 +                       da1p2
 +mirror/swap  COMPLETE  da0p3
 +                       da1p3
 + mirror/var  COMPLETE  da0p4
 +                       da1p4
 + mirror/tmp  COMPLETE  da0p5
 +                       da1p5
 + mirror/usr  COMPLETE  da0p6
 +                       da1p6</screen>
 +	  </listitem>
 +	</orderedlist>
 +    </sect2>
 +
 +    <sect2>
        <title>Troubleshooting</title>
  
        <sect3>
 @@ -382,9 +671,9 @@
  
  	<para>If the system boots up to a prompt similar to:</para>
  
 -	<programlisting>ffs_mountroot: can't find rootvp
 +	<screen>ffs_mountroot: can't find rootvp
  Root mount failed: 6
 -mountroot></programlisting>
 +mountroot></screen>
  
  	<para>Reboot the machine using the power or reset button.  At
  	  the boot menu, select option six (6).  This will drop the
 @@ -404,6 +693,45 @@
  	<para>in the kernel configuration file, rebuild and reinstall.
  	  That should remedy this issue.</para>
        </sect3>
 +
 +      <sect3>
 +	<title>System cannot find the boot device</title>
 +
 +	<para>In that case the system boot process would stop with a
 +		prompt similar to:</para>
 +
 +<screen><![ CDATA [Loader variables:
 +vfs.root.mountfrom=
 +vfs.root.mountfrom.options=
 +
 +Manual root filesystem specification:
 +  <fstype>:<device>  Mount <device> using filesystem <fstype>
 +                       eg. ufs:/dev/da0s1a
 +                       eg. cd9660:/dev/acd0
 +                       This is equivalent to: mount -t cd9660 /dev/acd0 /
 +
 +  ?                  List valid disk boot devices
 +  <empty line>       Abort manual input
 +
 +mountroot>]]></screen>
 +
 +	<para>You would get this message most probably if you forget
 +	  to add the correct boot device in
 +	  <filename>/boot/loader.conf</filename>. The hints given by the
 +	  system are clear. If your root device is
 +	  <devicename>/dev/mirror/root</devicename>, and the filesystem is
 +	  <literal>ufs</literal>, enter:</para>
 +
 +<screen>mountroot> <userinput>ufs:/dev/mirror/root</userinput></screen>
 +
 +	<para>The boot process would then continue:</para>
 + 
 +<screen>Trying to mount root from ufs:/dev/mirror/root</screen>
 +
 +	<para>When the system is up remember to edit your root device
 +	  in <filename>/boot/loader.conf</filename>.</para>
 +
 +      </sect3>
      </sect2>
  
      <sect2>
 
 -- 
 Anton Shterenlikht
 Room 2.6, Queen's Building
 Mech Eng Dept
 Bristol University
 University Walk, Bristol BS8 1TR, UK
 Tel: +44 (0)117 928 8233 
 Fax: +44 (0)117 929 4423
State-Changed-From-To: open->closed 
State-Changed-By: linimon 
State-Changed-When: Wed Mar 5 03:23:01 UTC 2014 
State-Changed-Why:  
Closed at submitter's request. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=136712 
>Unformatted:
