From nobody@FreeBSD.org  Fri Oct 11 13:55:16 2002
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 3C15F37B401
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 11 Oct 2002 13:55:16 -0700 (PDT)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id D625C43E7B
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 11 Oct 2002 13:55:15 -0700 (PDT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.6/8.12.6) with ESMTP id g9BKtF7R017732
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 11 Oct 2002 13:55:15 -0700 (PDT)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.6/8.12.6/Submit) id g9BKtFqQ017731;
	Fri, 11 Oct 2002 13:55:15 -0700 (PDT)
Message-Id: <200210112055.g9BKtFqQ017731@www.freebsd.org>
Date: Fri, 11 Oct 2002 13:55:15 -0700 (PDT)
From: Tim Kientzle <kientzle@acm.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Rationale for Upgrade Sequence
X-Send-Pr-Version: www-1.0

>Number:         43941
>Category:       docs
>Synopsis:       document the Rationale for Upgrade Sequence (e.g. why the "historical" build sequence is deprecated)
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    keramida
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Fri Oct 11 14:00:07 PDT 2002
>Closed-Date:    Sun Feb 01 04:38:11 UTC 2009
>Last-Modified:  Sun Feb 01 04:38:11 UTC 2009
>Originator:     Tim Kientzle
>Release:        
>Organization:
>Environment:
>Description:
Mark Linimon suggested that I "get this exact reasoning added to the official
docs, it's the clearest explanation I've seen yet."  This was in response to
a posting I made to freebsd-stable explaining some of the rationale behind the
buildworld/buildkernel/installkernel/installworld sequence.  I've attached the
text of my posting below.  I hope you find it useful.


>How-To-Repeat:
      
>Fix:
The following is an edited version of
my post to FreeBSD-stable mailing
list on 11 Oct, 2002, in response to a question
about why the "old" build sequence was no longer
considered sufficient:

------------------------------------------

Problem:  You're currently running an old system, consisting
of old compiler, old kernel, old world, and old configuration files.
(By "world" here, I mean the core system binaries, libraries,
and programming files.  The compiler is part of "world", but
has a few special concerns, which is why I've named it separately
here.)  You have source code for a new system and want to compile
and install it over the old system.  How should this be done?

This problem is a bit more subtle than it might seem, and the
FreeBSD developers have found it necessary over the years to
change the recommended approach fairly dramatically as new
kinds of unavoidable dependencies come to light.  The following
describes the rationale behind the current update sequence
(being used by 5.0, for example).

Any successful update sequence must deal with the
following issues:

* The old compiler might not be able to compile
  the new kernel.  (Old compilers sometimes have bugs.)
  So, the new kernel should be built with the new compiler.
  In particular, the new compiler must be built before
  the new kernel is built.  This does not necessarily
  mean that the new compiler must be _installed_ before
  building the new kernel, however.

* The new world might rely on new kernel features.
  So, the new kernel must be installed before the new world
  is installed.

These first two issues are the basis for the
core buildworld/buildkernel/installkernel/installworld
sequence that I'll describe in a moment.  They
are not the only concerns, however:

* The old world might not run correctly on the new kernel,
  so you must install the new world immediately
  upon installing the new kernel.

* Some configuration changes must be done before
  the new world is installed, but others might break
  the old world.  Hence, two different configuration
  upgrade steps are generally needed.

* For the most part, the update process only replaces
  or adds files; existing old files are not deleted.
  In a few cases, this can cause problems.  As a result,
  the update procedure will sometimes specify certain files
  that should be manually deleted at certain steps.
  This may or may not be automated in the future.

These concerns have led to the following recommended
sequence.  Note that the detailed sequence for
particular updates may require additional steps,
but this core process should remain unchanged for some
time:

1) make buildworld

   This first compiles the new compiler and
   a few related tools, then uses the new compiler
   to compile the rest of the new world.  The result
   ends up in /usr/obj.

2) make buildkernel

   Unlike the older config/make approach,
   this uses the _new_ compiler residing in
   /usr/obj.  This protects you against compiler/kernel
   mismatches.

3) make installkernel

   Place the new kernel and kernel modules onto the disk.

4) Reboot to single user mode.

   Single user mode minimizes problems from
   updating software that's already running.
   It also minimizes any problems from
   running the old world on a new kernel.
   (Curiously, when moving from 4.x to 5.0,
   you're still running the old kernel here,
   because the new kernel is in a different
   place and the boot loader hasn't been
   updated yet.)

5) mergemaster -p

   This does some initial configuration file updates
   in preparation for the new world.

6) make installworld

   Copies the world from /usr/obj.
   You now have a new kernel and new world on disk.

7) mergemaster

   Now you can update the remaining configuration files,
   since you have a new world on disk.

8) reboot

   A full machine reboot is needed now
   to load the new kernel and new world
   with new configuration files.


Note that if you're upgrading from, say,
4.6.2 to 4.7, then this procedure may not
be absolutely necessary, since you're
unlikely to run into serious mismatches
between compiler/kernel/world/configuration.
The older approach of "make world" followed
by building and installing a new kernel
might work well enough for minor updates.

But, when upgrading across major releases,
people who don't follow this procedure
should expect some problems.

Also, note that many upgrades (4.x to 5.0,
for instance) may require specific additional
steps (renaming or deleting specific files
prior to installworld, for instance).  Read
the UPDATING file carefully, especially at
the end, where the currently recommended upgrade
sequence is spelled out.

This procedure has evolved over time
as the developers have found it impossible
to completely prevent certain kinds of
mismatch problems.  Hopefully, the current
procedure will remain stable for a long time.

Note that upgrading from 3.x or earlier
is a bit trickier; read UPDATING carefully.


>Release-Note:
>Audit-Trail:

From: Chris Pepper <pepper@reppep.com>
To: Tim Kientzle <kientzle@acm.org>
Cc: freebsd-gnats-submit@FreeBSD.org, dougb@FreeBSD.org
Subject: Re: docs/43941: Rationale for Upgrade Sequence
Date: Fri, 11 Oct 2002 17:12:30 -0400

 At 1:55 PM -0700 2002/10/11, Tim Kientzle wrote:
 
 >  >Description:
 >Mark Linimon suggested that I "get this exact reasoning added to the 
 >official docs, it's the clearest explanation I've seen yet."  This 
 >was in response to a posting I made to freebsd-stable explaining 
 >some of the rationale behind the 
 >buildworld/buildkernel/installkernel/installworld sequence.  I've 
 >attached the text of my posting below.  I hope you find it useful.
 
 >* The new world might rely on new kernel features.
 >   So, the new kernel must be installed before the new world
 >   is installed.
 
 	It's worth mentioning that installing a new broken kernel is 
 testable and fixable, while installing a new world is much uglier. 
 This is another reason to installkernel, reboot, and installworld if 
 nothing blows up.
 
 >These concerns have led to the following recommended
 >sequence.  Note that the detailed sequence for
 >particular updates may require additional steps,
 >but this core process should remain unchanged for some
 >time:
 >
 >1) make buildworld
 >
 >    This first compiles the new compiler and
 >    a few related tools, then uses the new compiler
 >    to compile the rest of the new world.  The result
 >    ends up in /usr/obj.
 >
 >2) make buildkernel
 >
 >    Unlike the older config/make approach,
 >    this uses the _new_ compiler residing in
 >    /usr/obj.  This protects you against compiler/kernel
 >    mismatches.
 >
 >3) make installkernel
 >
 >    Place the new kernel and kernel modules onto the disk.
 >
 >4) Reboot to single user mode.
 >
 >    Single user mode minimizes problems from
 >    updating software that's already running.
 >    It also minimizes any problems from
 >    running the old world on a new kernel.
 >    (Curiously, when moving from 4.x to 5.0,
 >    you're still running the old kernel here,
 >    because the new kernel is in a different
 >    place and the boot loader hasn't been
 >    updated yet.)
 >
 >5) mergemaster -p
 >
 >    This does some initial configuration file updates
 >    in preparation for the new world.
 
 	mergemaster's documentation says it should run with -p before 
 buildworld, so if this is intended to be "official", mergemaster's 
 docco should be updated.
 
 	From mergemaster(8):
 >      -p          Pre-buildworld mode.  Compares only files known to be essen-
 >                  tial to the success of {build|install}world, including
 >                  /etc/make.conf.
 
 >[www:~] root# mergemaster --help
 >mergemaster version 1.6.2.14
 >Usage: mergemaster [-scrvahipC] [-m /path]
 >          [-t /path] [-d] [-u N] [-w N] [-D /path]
 >Options:
 >   -s  Strict comparison (diff every pair of files)
 >   -c  Use context diff instead of unified diff
 >   -r  Re-run on a previously cleaned directory (skip temproot creation)
 >   -v  Be more verbose about the process, include additional checks
 >   -a  Leave all files that differ to merge by hand
 >   -h  Display more complete help
 >   -i  Automatically install files that do not exist in destination directory
 >   -p  Pre-buildworld mode, only compares crucial files
 >   -C  Compare local rc.conf variables to the defaults
 >   -m /path/directory  Specify location of source to do the make in
 >   -t /path/directory  Specify temp root directory
 >   -d  Add date and time to directory name (e.g., /var/tmp/temproot.1011.17.11)
 >   -u N  Specify a numeric umask
 >   -w N  Specify a screen width in columns to sdiff
 >   -D /path/directory  Specify the destination directory to install files to
 
 
 						Chris Pepper
 -- 
 Chris Pepper:               <http://www.reppep.com/~pepper/>
 Rockefeller University:     <http://www.rockefeller.edu/>

From: Tim Kientzle <kientzle@acm.org>
To: Chris Pepper <pepper@reppep.com>
Cc: freebsd-gnats-submit@FreeBSD.org, dougb@FreeBSD.org,
	Warner Losh <imp@village.org>
Subject: Re: docs/43941: Rationale for Upgrade Sequence
Date: Fri, 11 Oct 2002 15:00:30 -0700

 The current UPDATING file
 
 puts the 'mergemaster -p' step
 after installkernel/reboot and before
 installworld.
 
 Warner Losh (who maintains UPDATING)
 should be able to say whether UPDATING
 or mergemaster(8) is correct.
 
 Tim Kientzle
 
 Re <http://www.freebsd.org/cgi/query-pr.cgi?pr=43941>,
 Chris Pepper observes:
 
 >     mergemaster's documentation says it should run with -p before 
 > buildworld, so if this is intended to be "official", mergemaster's docco 
 > should be updated.
 > 
 >     From mergemaster(8):
 > 
 >>      -p          Pre-buildworld mode.  Compares only files known to be 
 >> essen-
 >>                  tial to the success of {build|install}world, including
 >>                  /etc/make.conf.
 
 
 

From: "M. Warner Losh" <imp@bsdimp.com>
To: kientzle@acm.org
Cc: pepper@reppep.com, freebsd-gnats-submit@FreeBSD.org,
	dougb@FreeBSD.org, imp@village.org
Subject: Re: docs/43941: Rationale for Upgrade Sequence
Date: Fri, 11 Oct 2002 16:15:23 -0600 (MDT)

 In message: <3DA749FE.8010407@acm.org>
             Tim Kientzle <kientzle@acm.org> writes:
 : The current UPDATING file
 : 
 : puts the 'mergemaster -p' step
 : after installkernel/reboot and before
 : installworld.
 : 
 : Warner Losh (who maintains UPDATING)
 : should be able to say whether UPDATING
 : or mergemaster(8) is correct.
 
 mergemaster -p isn't needed for buildworld, but is needed for
 installworld.  One can run it before buildworld without ill effect, as
 far as I know, but it isn't required.  /etc/make.conf is empty except
 for comments by default, so updating it with mergemaster likely won't
 make any difference.
 
 Warner
Responsible-Changed-From-To: freebsd-doc->keramida 
Responsible-Changed-By: keramida 
Responsible-Changed-When: Wed Jan 16 07:08:50 UTC 2008 
Responsible-Changed-Why:  
Grab PR; I'll try to integrate the text of Tim to the Handbook 

http://www.freebsd.org/cgi/query-pr.cgi?pr=43941 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: docs/43941: commit references a PR
Date: Sun,  1 Feb 2009 04:25:31 +0000 (UTC)

 keramida    2009-02-01 04:25:22 UTC
 
   FreeBSD doc repository
 
   Modified files:
     en_US.ISO8859-1/books/handbook/cutting-edge chapter.sgml 
   Log:
   Add an explanation of the reasons that buildworld, buildkernel,
   installkernel, reboot, ... is a good idea. Before describing
   every step in detail, we let the reader know "why" so that they
   can later appreciate the "how" a bit more, and feel less confused
   about the whole dance.
   
   PR:             docs/43941
   Submitted by:   kientzle
   Reviewed by:    trhodes
   
   Revision  Changes    Path
   1.240     +184 -3    doc/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.sgml
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: keramida 
State-Changed-When: Sun Feb 1 04:37:33 UTC 2009 
State-Changed-Why:  
It took me a while, but I finally managed to SGMLify this. 
Committed as revision 1.240 of cutting-edge/chapter.sgml. 

Thanks! :) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=43941 
>Unformatted:
