From nobody@FreeBSD.org  Fri Jan  4 21:29:54 2002
Return-Path: <nobody@FreeBSD.org>
Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21])
	by hub.freebsd.org (Postfix) with ESMTP id 2EAC737B417
	for <freebsd-gnats-submit@FreeBSD.org>; Fri,  4 Jan 2002 21:29:54 -0800 (PST)
Received: (from nobody@localhost)
	by freefall.freebsd.org (8.11.6/8.11.6) id g055TsC40649;
	Fri, 4 Jan 2002 21:29:54 -0800 (PST)
	(envelope-from nobody)
Message-Id: <200201050529.g055TsC40649@freefall.freebsd.org>
Date: Fri, 4 Jan 2002 21:29:54 -0800 (PST)
From: Craig Carey <research@ijs.co.nz>
To: freebsd-gnats-submit@FreeBSD.org
Subject: RELENG_4 won't makeworld; bsd.dep.mk, Makefile.inc1 bugs; "Linux ELF" need of kernel
X-Send-Pr-Version: www-1.0

>Number:         33567
>Category:       misc
>Synopsis:       RELENG_4 won't makeworld; bsd.dep.mk, Makefile.inc1 bugs; "Linux ELF" need of kernel
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Jan 04 21:30:00 PST 2002
>Closed-Date:    Fri Aug 09 09:38:48 PDT 2002
>Last-Modified:  Mon Jan 20 02:40:01 PST 2003
>Originator:     Craig Carey
>Release:        RELENG_4_3
>Organization:
>Environment:
An AMD Duron PC
>Description:
      


Notes on issues that arose during a buildworld of RELENG_4
 FreeBSD of 4 Jan 2002.

Background:

    I upgraded from v4.3 to v5. The nonbootable kernel had the known
    incompatability of syscons and pcvt, where none of these would
    boot: neither, one, and both. A cvsup to v4.4 led to a buildworld
    failing with a lot of errors. With v4 (RELENG_4), the buildworld
    succeeded, with errors.

    My LINT config file was trimmed down. I could provide it if that
    was requested.

The report is mainly about fixes needed to the files:

  *    /usr/src/Makefile.inc1     ("src-base" )

  *    /src/share/mk/bsd.dep.mk   ("src-share")

    All the following comments are about a FreeBSD system in a PC that
    had all of the src sources cvsupped to RELENG_4 on 4 January 2002.

Problem:
    The msdosfs directory had shifted and buildworld crashed due to
    the 'local' mtree file not being updated. I don't known why the
    file was not updated.

    Aside: locate mtree | xargs -n1 egrep -H msdosfs

Problem:
    File "stdint.h" got automatically erased by cvsup when moving to
    RELENG_4. The software, XFree86 4.x, needs that header file.

Problem:
    The "/machine" include file directory has gone, so files including
    file ansi.h can't be found during the buildworld.

    Fix: A fix was to do this:
      cd /usr/src/sys ; ln -s i386/include/  machine

Problem:
    Many of crashes of "make -DNOCLEAN buildworld" were fixable by
    adding to the top of the file

      /usr/share/mk/bsd.dep.mk

    this line:

      CFLAGS+= -I/usr/src/sys -I/usr/src/include \
               -I/usr/src/sys/i386/include -I/usr/src/sys/sys

    This is a problem with the bsd.dep.mk file. Those 4 include paths
    seemed to be the minimum that would allow success.

Problem:
    Buildworld of RELENG_4 crashed the make of texinfo. [For
    RELENG_4_4 it did start fail on the build of yacc].
    Detail:

      cd /usr/src ; export MAKEFLAGS=""; make -DNOCLEAN buildworld

      /usr/src/ .. /contrib/texinfo/info/terminal.c:36: termcap.h:
                                              No such file or directory
    Fix:
      Edit  /usr/src/Makefile.inc1
      and remove  "gnu/usr.bin/texinfo".

Problem:
    Because of the seemingly needed editing of
    "/usr/src/Makefile.inc1", makeworld has to be run twice.

Problem:
    Libncurses would not build. A file was missing. So texinfo would
    not install, and from /usr/src/Makeinfo.inc1, the text
    "lib/libncurses" was deleted.

    Libncurses also needed a "cpp/" directory that was not on the
    filesystem and that was not obtained by cvsup of the RELENG_4 src
    files.

Problem:
    The make of the "file(1)" program failed. One of the errors was:

      /usr/src/[..]/contrib/file/file.h:123: stdio.h: No such file or directory

    Unlike most of the other software, "file"'s Makefile specified the
    include file and got it wrong.

    (An attempt to correct the include file,
    /usr/src/contrib/file/Makefile by adding this line
      DEFS = -DHAVE_CONFIG_H -I/usr/src/include -I/usr/src/sys \
                                    -I/usr/src/sys/sys -I. -I$(srcdir) -I.
    led to the error:
      file.h:172: conflicting types for `sys_errlist'
      /usr/src/include/stdio.h:225: previous declaration of `sys_errlist'
    )

    Fix:
      Edit  /usr/src/Makefile.inc1
      and remove  "usr.bin/file".

Issue:
    Perl would not build.

    Due to no texinfo in the first pass [the Makefile.inc1 file tries
    to build in a wrong order], the 'info' files were not be built.

    I edited Makefile.inc1 and removed the text "lib/libcom_err/doc".

Another problmem: buildkernel of RELENG_4 does not boot:

    The  /usr/src/sys/i386/conf/<LINT>  file has edited so that all
    the linux-features lines commented out. The boot dies with the
    last lines being:

      start_init: trying /sbin/init
      Linux ELF exec handler installed

    My RELENG_4 kernel is maybe unable to run "/modules/linux.ko" or
    something.

    Removing COMPAT_LINUX (or whatever it is called) led to a compile
    error during a buildkernel (I lost the details on that).

    When I booted from a custom v4.3 kernel and the online 4.4 kernel
    that came from here

      ftp://ftp2.freebsd.org/pub/FreeBSD/releases/i386/4.4-RELEASE/kernel

    then it was the case that kldload produced a <<can't "exec"
    linux.ko>> error. Maybe the cvsupped-RELENG_4 custom kernel is
    incompatible with the linux.ko file that was created at the same
    time by buildkernel.

    (My notes show that this appeared. I have no recollection of
    whether this is output from a non-booting v4, 4.4, or v5,
    customised kernel. The booting hung after producing this:

      vmioing
      start_init: trying /sbin/init
      getblk: vmioing
      panic biodone
      zero vnode refcount 
   )

Suggestions

    By default the console can't be configured to show 132 columns.
    That is the case for the release 4.4-RELEASE kernel. Can that be
    rechecked?. It seems unsafe to expect beginners to spend days or
    weeks debugging FreeBSD RELENG_4 scripts viewing only 80 columns
    and very stretched fonts. A newcomer might want to get the
    soundcard to run, and want 132 columns so that the console windows
    seem not too stretched, and get ext2 accessible, and get a fast
    accurate PS/2 mouse that does not seem to be very very much worse
    than a Windows 95 PS/2 mouse (40 samples per second, and 60 in
    NT/2000 apparently; 'man -k psm'). Getting bootable kernel seems
    to be difficult with the 4 & 4.4 src files.

    Boot messages now appear as reverse video white text over red. The
    LINT file specified white on black and it was overridden.

    I added "set -x" to the script "/etc/rc" and that script was
    seemingly not called by my RELENG_4 4 Janurary kernel (it died
    before calling /etc/rc most probably.

 

>How-To-Repeat:
      
>Fix:
      Some of the fixes are contained in the Description section.
>Release-Note:
>Audit-Trail:

From: Craig Carey <research@ijs.co.nz>
To: freebsd-gnats-submit@FreeBSD.org, research@ijs.co.nz
Cc:  
Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk,
  Makefile.inc1 bugs; "Linux ELF" need of kernel
Date: Thu, 10 Jan 2002 18:16:55 +1300

 Retraction of some parts of my report:
 
 The problem with hanging after "/sbin/init" (my report says a
 problem was with ELF Linux) was fixable by making some alterations to
 the LINT file. My LINT file at the time had parts of a version 5 LINT
 file in it (mainly the device lines but the hints lines were all
 commented out since v4 "config" won't accept those). If I tracked down
 the findings then maybe they would be ignored. Certainly it is a
 problem others could later also have. I got the LINUX ELF bugs to
 go away without finding out what it was.
 
 sbin/init diagnostic errors messages seem to not inform properly.
 
 I got the colour was fixed and /etc/rc ran OK.
 
 That PS/2 mouse problems are real. Some is due to XFree86 code needing
 a bug fix and FreeBSD allows configuration of the mouse. I
 
 ---
 
 I wrote on 132 columns capability not being the default. That refers
 to the absence of a line containing the text "VGA_WIDTH90" in the
 GENERIC 'LINT' file of both v5 (HEAD) and RELENG_4 FreeBSD LINT files.
 This line:
 
     options         VGA_WIDTH90             # support 90 column modes
 
 An improvement may be to add the line to the GENERIC files.
 
 The appearance of VGA_WIDTH90:
 http://www.FreeBSD.org/cgi/query-pr.cgi?pr=i386/7510
 A 22 June 1999 record of VGA_WIDTH90 being added:
 http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/conf/NOTES.diff?r1=1.607&r2=1.608&f=h
 
 That might not be a bug, in addition to perhaps being something can
 be unresolved when the report is closed (if closed).
 
 Craig Carey
 

From: "Crist J . Clark" <cjc@FreeBSD.ORG>
To: Craig Carey <research@ijs.co.nz>
Cc: freebsd-gnats-submit@FreeBSD.ORG
Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk, Makefile.inc1 bugs; "Linux ELF" need of kernel
Date: Thu, 10 Jan 2002 17:24:32 -0800

 On Fri, Jan 04, 2002 at 09:29:54PM -0800, Craig Carey wrote:
 [snip]
 
 > Background:
 > 
 >     I upgraded from v4.3 to v5. The nonbootable kernel had the known
 >     incompatability of syscons and pcvt, where none of these would
 >     boot: neither, one, and both. A cvsup to v4.4 led to a buildworld
 >     failing with a lot of errors. With v4 (RELENG_4), the buildworld
 >     succeeded, with errors.
 
 
 > Problem:
 >     The msdosfs directory had shifted and buildworld crashed due to
 >     the 'local' mtree file not being updated. I don't known why the
 >     file was not updated.
 > 
 >     Aside: locate mtree | xargs -n1 egrep -H msdosfs
 
 It had "shifted?"
 
 > Problem:
 >     File "stdint.h" got automatically erased by cvsup when moving to
 >     RELENG_4. The software, XFree86 4.x, needs that header file.
 
 XFree86-4 runs fine on STABLE which, as you say, has no stdint.h.
 
 > Problem:
 >     The "/machine" include file directory has gone, so files including
 >     file ansi.h can't be found during the buildworld.
 > 
 >     Fix: A fix was to do this:
 >       cd /usr/src/sys ; ln -s i386/include/  machine
 
 Looks like your kernel build was messed up.
 
 > Problem:
 >     Many of crashes of "make -DNOCLEAN buildworld" were fixable by
 >     adding to the top of the file
 
 Ah, and this looks like why. You can't do '-DNOCLEAN' between source
 code updates and expect things to work. I'd actually recommend nuking
 the whole object tree,
 
   # rm -rf /usr/obj/usr
 
 I cannot replicate any of the failures. On a CURRENT box, RELENG_4
 builds fine for me.
 -- 
 "It's always funny until someone gets hurt. Then it's hilarious."
 
 Crist J. Clark                     |     cjclark@alum.mit.edu
                                    |     cjclark@jhu.edu
 http://people.freebsd.org/~cjc/    |     cjc@freebsd.org
State-Changed-From-To: open->feedback 
State-Changed-By: sheldonh 
State-Changed-When: Tue Jan 29 04:45:39 PST 2002 
State-Changed-Why:  
It's hard to find outstanding issues in the audit trail of this PR. 
That's part of the problem associated with reporting more than one 
problem in a single PR. 

Given that the recommended procedure for downgrading from CURRENT to 
STABLE is a binary install / upgrade, could you itemize the outstanding 
problems that need attention? 

Please copy your feedback to <bug-followup@freebsd.org>, using the 
subject line of this message. 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=33567 

From: Craig Carey <research@ijs.co.nz>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk,
  Makefile.inc1 bugs; "Linux ELF" need of kernel
Date: Wed, 06 Feb 2002 08:21:23 +1300

 At 02.01.29 04:45 -0800 Tuesday, sheldonh@FreeBSD.org wrote:
 
 ------------------
 ...
 Received: (from sheldonh@localhost)
 	by freefall.freebsd.org (8.11.6/8.11.6) id g0TCjrt98270;
 	Tue, 29 Jan 2002 04:45:53 -0800 (PST)
 	(envelope-from sheldonh)
 *Date: Tue, 29 Jan 2002 04:45:53 -0800 (PST)
 *From: <sheldonh@FreeBSD.org>
 Message-Id: <200201291245.g0TCjrt98270@freefall.freebsd.org>
 *To: research@ijs.co.nz, sheldonh@FreeBSD.org,
 *	freebsd-bugs@FreeBSD.org
 *Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk, Makefile.inc1
 *           bugs; "Linux ELF" need of kernel
 ------------------
 
  >Synopsis: RELENG_4 won't makeworld; bsd.dep.mk, Makefile.inc1 bugs; "Linux ELF" need of kernel
  >
  >State-Changed-From-To: open->feedback
  >State-Changed-By: sheldonh
  >State-Changed-When: Tue Jan 29 04:45:39 PST 2002
  >State-Changed-Why:
  >It's hard to find outstanding issues in the audit trail of this PR.
  >That's part of the problem associated with reporting more than one
  >problem in a single PR.
  >
  >Given that the recommended procedure for downgrading from CURRENT to
  >STABLE is a binary install / upgrade, could you itemize the outstanding
  >problems that need attention?
 
 FreeBSD experts download hundreds of MB of files instead of editing up
 buildworld scripts to get them past a string of crashes?. I would
 recommend that advise on recommended downgrade methods be worded so that
 it does not apply to FreeBSD experts that know about how to edit the
 *.mk files (something that I thought needed for the v5->v4 downgrade).
 
 Perhaps that sentence starting with "Given that ..." is actually saying
 that most parts of my bug report are already disregarded. It is not that
 clear. It would unreasonable to be frank, to suppose that FreeBSD experts
 should themselves follow the 'alleged to exist' recommendation that
 apparently would apply against the masses that would fail in their
 attempt to go from v4 to v5.
 
 Such a closing would misses the aspect that a mere few added lines could
 be all that it takes to have less person hours being wasted. They may
 be a failure in the bug handling decision there. But I previously saw
 that Mr Clark was edging onto that. He and I were writing privately,
 including on bits around law. Messages never appeared at the report
 system.
 
 I find the sentence "Given that the .... attention", is very much too
 subtly worded, if indeed it is attempting to close most of the report.
 
 The dep.bsd.mk problems was about missing include file paths, and it
 was suggested that some recommendation says to get binary files which
 maybe makes no change to the include files. It is far from clear that
 following the recommendation does not lead to exactly the same problems.
 
 In any case, a quick closing on technical grounds can be unsound when
 the bugs are small and tiny in number and partly identified, and the
 inconvenience that not fixing them could cause is large. What happened
 to Mr Clark who was handling this. Since he and I did communicate over
 law in 'misc/33567' (and that has not made it into the bug system's
 archives) then there could have been a different outcome not that I
 can be sure if a "not recommended way was used" thing happened here.
 
 
  >
  >Please copy your feedback to <bug-followup@freebsd.org>, using the
  >subject line of this message.
  >
  >http://www.FreeBSD.org/cgi/query-pr.cgi?pr=33567
 
 
 I never found out why the report could not be reproduced. It is plain
 that many would not want to but there was a report that reproducing
 the problems of 33567 was attempted.
 
 I remind that I declared that these lines should be added to /usr/share/mk/bsd.dep.mk :
 
 
       CFLAGS+= -I/usr/src/sys -I/usr/src/include \
                -I/usr/src/sys/i386/include -I/usr/src/sys/sys
 
 
 I added those lines and buildworld would proceed successfully except
 for resolvable problems with "/usr/src/Makefile.inc1". Thus even some
 testing of the described bug fixes was done.
 
 I will overlook the fact that a technical arguments was used.
 There are many ways to close this FreeBSD bug report.
 
 
 Craig Carey
 

From: Craig Carey <research@ijs.co.nz>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk,
  Makefile.inc1 bugs; "Linux ELF" need of kernel
Date: Wed, 06 Feb 2002 10:36:03 +1300

 To say that document writing people limit the number of
 recommended downgrade paths could be a referring to an
 adaptation of those people, to a failure to get the bugs out.
 This report hopefully ought not be closed on the basis of some
 circular argument.
 
 The recommendation (if any) possibly mainly applies to just
 those people that don't know how to fix the buildworld bugs. A
 recommendation to avoid downgrades is like a clue that they are
 there, so maybe a partial step towards satisfying members of
 FreeBSD-Current is to remove bugs. A recommendation saying a
 path ought be avoided is not the same as saying that problems
 with it ought be left unfixed. If there was a principle then
 can that be a lot more clearly stated.
 
 I request that the bug report be closed on the grounds that I
 have not been using FreeBSD for over a month. The next
 message could clarify the decision (perhaps it would have new
 material in it).
 
 
 Craig Carey
 

From: Giorgos Keramidas <keramida@ceid.upatras.gr>
To: Craig Carey <research@ijs.co.nz>
Cc: bug-followup@freebsd.org
Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk, Makefile.inc1 bugs; "Linux ELF" need of kernel
Date: Thu, 7 Feb 2002 02:53:37 +0200

 On 2002-02-05 11:30, Craig Carey wrote:
 > FreeBSD experts download hundreds of MB of files instead of editing up
 > buildworld scripts to get them past a string of crashes?. I would
 > recommend that advise on recommended downgrade methods be worded so that
 > it does not apply to FreeBSD experts that know about how to edit the
 > *.mk files (something that I thought needed for the v5->v4 downgrade).
 >  
 > Perhaps that sentence starting with "Given that ..." is actually saying
 > that most parts of my bug report are already disregarded. It is not that
 > clear. It would unreasonable to be frank, to suppose that FreeBSD experts
 > should themselves follow the 'alleged to exist' recommendation that
 > apparently would apply against the masses that would fail in their
 > attempt to go from v4 to v5.
 
 Actually, this is the 'safe' method of downgrading your FreeBSD installation.
 A quick search in groups.google.com came up with this old posting from Kris
 Kennaway that shows this has been the recommended way for quite a while :-)
 
 http://groups.google.com/groups?selm=20020124101941.E52223_xor.obsecurity.org%40ns.sol.net&output=gplain
 
 -- 
 Giorgos Keramidas . . . . . . . . . keramida@{ceid.upatras.gr,freebsd.org}
 FreeBSD Documentation Project . . . http://www.freebsd.org/docproj/
 FreeBSD: The power to serve . . . . http://www.freebsd.org/
State-Changed-From-To: feedback->closed 
State-Changed-By: schweikh 
State-Changed-When: Fri Aug 9 09:34:01 PDT 2002 
State-Changed-Why:  
Closed at request of originator. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=33567 

From: Craig Carey <research@ijs.co.nz>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: misc/33567: RELENG_4 won't makeworld; bsd.dep.mk,
  Makefile.inc1 bugs; "Linux ELF" need of kernel
Date: Mon, 20 Jan 2003 23:23:48 +1300

 The now closed #misc/33567 problem report of Jan 2002, is now better
 understood by me. The report said that downgrading from version 5 of
 FreeBSD to version 4, i.e. doing a makeworld, was full of failed
 compiles.
 
 But report #33567 has little ability to show that since my
 /usr/src/contrib/ directory was largely incomplete.
 
 Also the Linux Emulator would not run because the kernel option
     DEBUG_LOCKS
 was enabled. The documentation in the LINT does not say that option
 kills the Linuxulator. (A missing vn_something symbol for linux.ko).
 
 
 This webpage says FreeBSD can get by with 350MB of hard disk space:
 
     http://www.freebsd.org/relnotes/4-STABLE/installation/i386/index.html
 
 It says:
 
        "a more realistic minimum is on the order of 250-350MB"
 
 That 350MB is too small. 3.5GB seems much more realistic.
 
 Omitting the /usr/src/contrib directory saved about 100MB +/- 20MB,
 and got my FreeBSD system that would not compile correctly, nearer
 to the suggested size.
 
 Eventually I created another FFS partition inside of a MSDOS
 extended partition, but only after quitting FreeBSD for Linux, for
 a few moments.
 
 
 Craig Carey
 
 
 
>Unformatted:
