From nobody@FreeBSD.org  Wed Mar 17 09:37:41 2004
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 6638C16A4CE
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 17 Mar 2004 09:37:41 -0800 (PST)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 61E4B43D1F
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 17 Mar 2004 09:37:41 -0800 (PST)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.12.10/8.12.10) with ESMTP id i2HHbf72064515
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 17 Mar 2004 09:37:41 -0800 (PST)
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.12.10/8.12.10/Submit) id i2HHbfUf064514;
	Wed, 17 Mar 2004 09:37:41 -0800 (PST)
	(envelope-from nobody)
Message-Id: <200403171737.i2HHbfUf064514@www.freebsd.org>
Date: Wed, 17 Mar 2004 09:37:41 -0800 (PST)
From: mark wolgemuth <mark@node.to>
To: freebsd-gnats-submit@FreeBSD.org
Subject: exec of linux binary in pxeboot diskless nfs root system causes panic 
X-Send-Pr-Version: www-2.3

>Number:         64378
>Category:       kern
>Synopsis:       exec of linux binary in pxeboot diskless nfs root system causes panic
>Confidential:   no
>Severity:       critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Mar 17 09:40:20 PST 2004
>Closed-Date:    Wed Mar 17 19:05:10 PST 2004
>Last-Modified:  Wed Mar 17 19:10:17 PST 2004
>Originator:     mark wolgemuth
>Release:        5.2.1-RELEASE-p1
>Organization:
employease, inc.
>Environment:
FreeBSD nonuts 5.2.1-RELEASE-p1 FreeBSD 5.2.1-RELEASE-p1 #9: Tue Mar 16 12:45:00 EST 2004     root@demon.tek.eease.com:/usr/src/sys/i386/compile/NETBOOT  i386
>Description:
Kernel is loaded over pxeboot + nfs. Root and usr are mounted over NFS from Netapp.
Modules linux and linprocfs are loaded via loader.conf.
System comes up ok, but execution of a linux binary, eg. "/compat/linux/sbin/ldconfig" causes panic. Instruction ptr is at "Xpage".
I have union mounted /compat/linux under a read/write mounted fs (md in this case) so that /compat/linux/etc/ld.so.cache can be written, to rule out that problem. Since /compat/linux is writeable, so is /compat/linux/var also.
>How-To-Repeat:
Boot a diskless i386 system using pxeboot and root and usr on read-only nfs.
Load linux kernel module. Try to run /compat/linux/ldconfig.
>Fix:
      
>Release-Note:
>Audit-Trail:

From: "mark" <mark@employease.com>
To: <freebsd-gnats-submit@FreeBSD.org>, <mark@node.to>
Cc:  
Subject: Re: kern/64378: exec of linux binary in pxeboot diskless nfs root system causes panic
Date: Wed, 17 Mar 2004 20:18:38 -0500

 Tried mounting /usr/compat directly to a memory disk, and untarring a
 tarball of /usr/compat install there.
 Same error.
 
 Tried making a ufs filesystem on a local disk (this netboot box has one in
 it).
 Mounted /usr/compat to disk and untarred tarball onto it.
 Same error.
 
 So it doesn't like like has to do with r/w perms or setting "text file busy
 flag".
 
 --mark
 
State-Changed-From-To: open->feedback 
State-Changed-By: kris 
State-Changed-When: Wed Mar 17 15:08:00 PST 2004 
State-Changed-Why:  
Feedback requested 

http://www.freebsd.org/cgi/query-pr.cgi?pr=64378 

From: Kris Kennaway <kris@obsecurity.org>
To: mark wolgemuth <mark@node.to>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/64378: exec of linux binary in pxeboot diskless nfs root system causes panic
Date: Wed, 17 Mar 2004 15:07:53 -0800

 On Wed, Mar 17, 2004 at 09:37:41AM -0800, mark wolgemuth wrote:
 
 > FreeBSD nonuts 5.2.1-RELEASE-p1 FreeBSD 5.2.1-RELEASE-p1 #9: Tue Mar 16 12:45:00 EST 2004     root@demon.tek.eease.com:/usr/src/sys/i386/compile/NETBOOT  i386
 > >Description:
 > Kernel is loaded over pxeboot + nfs. Root and usr are mounted over NFS from Netapp.
 > Modules linux and linprocfs are loaded via loader.conf.
 > System comes up ok, but execution of a linux binary, eg. "/compat/linux/sbin/ldconfig" causes panic. Instruction ptr is at "Xpage".
 > I have union mounted /compat/linux under a read/write mounted fs (md in this case) so that /compat/linux/etc/ld.so.cache can be written, to rule out that problem. Since /compat/linux is writeable, so is /compat/linux/var also.
 > >How-To-Repeat:
 > Boot a diskless i386 system using pxeboot and root and usr on read-only nfs.
 > Load linux kernel module. Try to run /compat/linux/ldconfig.
 
 Sounds like it could be a few things:
 
 * Stale kernel modules; they need to be built from the same sources as your kernel
 
 * Known unionfs bugs (see the manpage).  It's not clear whether you're
 using unionfs or the union mount option.
 
 * Something else :)
 
 Please obtain a debugging traceback of the panic as described in:
 
   http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug.html
 
 This is required in order to proceed with evaluating this PR.
 
 Kris
State-Changed-From-To: feedback->closed 
State-Changed-By: kris 
State-Changed-When: Wed Mar 17 19:04:43 PST 2004 
State-Changed-Why:  
Submitter reports problem was due to local kernel configuration 
changes 

http://www.freebsd.org/cgi/query-pr.cgi?pr=64378 

From: Kris Kennaway <kris@obsecurity.org>
To: freebsd-gnats-submit@FreeBSD.org
Cc:  
Subject: Re: kern/64378: exec of linux binary in pxeboot diskless nfs root system causes panic
Date: Wed, 17 Mar 2004 19:05:52 -0800

 Adding to audit trail
 
 ----- Forwarded message from mark <mark@node.to> -----
 
 X-Original-To: kkenn@localhost
 Delivered-To: kkenn@localhost.obsecurity.org
 Delivered-To: kris@freebsd.org
 Date: Thu, 18 Mar 2004 03:01:18 +0000 (GMT)
 From: mark <mark@node.to>
 Subject: Re: kern/64378: exec of linux binary in pxeboot diskless nfs root
  system causes panic
 In-reply-to: <20040317230753.GA70724@xor.obsecurity.org>
 X-X-Sender: mark@mister.mcgoonet.com
 To: Kris Kennaway <kris@obsecurity.org>
 Reply-To: mark@node.to
 X-UIDL: /ML!!mY9!!4V`!!GR+!!
 X-Bogosity: No, tests=bogofilter, spamicity=0.000000, version=0.16.4
 
 
 I did a bunch more investigation. I can't seem to get the nfs boot system
 to dump to its speficied dumpdev, despite it having an internal disk
 device for swap (not swap over nfs), with dumpon set.
 
 However:
 
 Stupidly, I realized I had tuned some memory settings on the kernel
 running over NFS, but not the system I was comparing it too.
 
 I had raised the "KVA_PAGES" limit above the default. Removing this caused
 the system to be able to execute linux binaries without panic.
 
 I had raised this limit because this system had panicked earlier under
 high proc count/ memory utilization. It's a 2x2.6ghz xeon with 2.5 gig
 ram. I saw a thread in the maillists saying that KVA_PAGES could be raised
 to prevent the panic I had seen earlier on systems that had a lot of RAM.
 
 However, it obviously had ramifications on linux emulation (no pun
 intended).
 
 I confess to a not totally clear understanding of the intracacies of
 tuning these vars:
 KVA_PAGES
 VM_KMEM_SIZE_MAX
 VM_KMEM_SIZE_SCALE
 
 I tried to do adequate research before submitting this problem as a bug,
 but I guess I didn't do enough.
 
 --mark
 
   mark@node.to  http://node.to/~mark   7123 3F7B 10EC 7122 2F8B
   http://node.to/keys/mark.asc         B474 B09D 6ED7 3FB0 09E8
 
 
 On Wed, 17 Mar 2004, Kris Kennaway wrote:
 
 > On Wed, Mar 17, 2004 at 09:37:41AM -0800, mark wolgemuth wrote:
 >
 > > FreeBSD nonuts 5.2.1-RELEASE-p1 FreeBSD 5.2.1-RELEASE-p1 #9: Tue Mar 16 12:45:00 EST 2004     root@demon.tek.eease.com:/usr/src/sys/i386/compile/NETBOOT  i386
 > > >Description:
 > > Kernel is loaded over pxeboot + nfs. Root and usr are mounted over NFS from Netapp.
 > > Modules linux and linprocfs are loaded via loader.conf.
 > > System comes up ok, but execution of a linux binary, eg. "/compat/linux/sbin/ldconfig" causes panic. Instruction ptr is at "Xpage".
 > > I have union mounted /compat/linux under a read/write mounted fs (md in this case) so that /compat/linux/etc/ld.so.cache can be written, to rule out that problem. Since /compat/linux is writeable, so is /compat/linux/var also.
 > > >How-To-Repeat:
 > > Boot a diskless i386 system using pxeboot and root and usr on read-only nfs.
 > > Load linux kernel module. Try to run /compat/linux/ldconfig.
 >
 > Sounds like it could be a few things:
 >
 > * Stale kernel modules; they need to be built from the same sources as your kernel
 >
 > * Known unionfs bugs (see the manpage).  It's not clear whether you're
 > using unionfs or the union mount option.
 >
 > * Something else :)
 >
 > Please obtain a debugging traceback of the panic as described in:
 >
 >   http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug.html
 >
 > This is required in order to proceed with evaluating this PR.
 >
 > Kris
 >
 
 ----- End forwarded message -----
>Unformatted:
