From buggy@kkcity.com.tw  Tue Oct 16 01:17:48 2001
Return-Path: <buggy@kkcity.com.tw>
Received: from www.kkcity.com.tw (www.kkcity.com.tw [210.59.230.230])
	by hub.freebsd.org (Postfix) with ESMTP id 5725037B407
	for <FreeBSD-gnats-submit@freebsd.org>; Tue, 16 Oct 2001 01:17:46 -0700 (PDT)
Received: by www.kkcity.com.tw (Postfix, from userid 1001)
	id 441C5B4654; Tue, 16 Oct 2001 16:16:51 +0800 (CST)
Message-Id: <20011016081651.441C5B4654@www.kkcity.com.tw>
Date: Tue, 16 Oct 2001 16:16:51 +0800 (CST)
From: justin@skysoft.com.tw
Reply-To: justin@skysoft.com.tw <buggy@kkcity.com.tw>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: pthread broken with KVA_PAGES=512
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         31310
>Category:       kern
>Synopsis:       pthread broken with KVA_PAGES=512
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 16 01:20:00 PDT 2001
>Closed-Date:    Mon Dec 17 14:58:12 PST 2001
>Last-Modified:  Mon Dec 17 15:00:22 PST 2001
>Originator:     Justin Chuang
>Release:        FreeBSD 4.4-STABLE i386
>Organization:
SkySoft Inc. Taiwan
>Environment:
FreeBSD www.kkcity.com.tw 4.4-RC FreeBSD 4.4-RC #0: Tue Oct  2 14:18:22 CST
2001

     root@www.kkcity.com.tw:/usr/src/sys/compile/KKCITY_KERNEL  i386

This is not correct version on which the bug occurs, which is 4.4-STABLE #2,
I can't boot into that kernel because it's a running server.

>Description:
KVA_PAGES in kernel config is set to 512, and I had 'make world'
to update /usr/lib.

When I run any program that use pthread (linked with libc_r), 
it prints:

 
Fatal error 'Cannot allocate red zone for initial thread' at line ? in file
/usr/src/lib/libc_r/uthread/uthread_init.c (errno = ?)

Abort trap (core dumped)
 

I didn't have this problem on 4.4-RC0 with ldscript.i386 and pmap.h 
patched as the following to enlarge kernel address space to 2G.

Index: conf/ldscript.i386
===================================================================
RCS file: /home/ncvs/src/sys/conf/ldscript.i386,v
retrieving revision 1.4
diff -u -r1.4 ldscript.i386
--- conf/ldscript.i386	2000/01/11 15:35:16	1.4
+++ conf/ldscript.i386	2001/04/02 16:07:18
@@ -6,7 +6,7 @@
 SECTIONS
 {
   /* Read-only sections, merged into text segment: */
-  . = 0xc0100000 + SIZEOF_HEADERS;
+  . = 0x80100000 + SIZEOF_HEADERS;
   .interp     : { *(.interp) 	}
   .hash          : { *(.hash)		}
   .dynsym        : { *(.dynsym)		}
Index: i386/include/pmap.h
===================================================================
RCS file: /home/ncvs/src/sys/i386/include/pmap.h,v
retrieving revision 1.70
diff -u -r1.70 pmap.h
--- i386/include/pmap.h	2000/11/30 01:53:02	1.70
+++ i386/include/pmap.h	2001/04/02 16:07:18
@@ -92,9 +92,9 @@
 #endif
 #ifndef NKPDE
 #ifdef SMP
-#define NKPDE			254	/* addressable number of page
tables/pde's */
+#define NKPDE			510	/* addressable number of page
tables/pde's */
 #else
-#define NKPDE			255	/* addressable number of page
tables/pde's */
+#define NKPDE			511	/* addressable number of page
tables/pde's */
 #endif	/* SMP */
 #endif


>How-To-Repeat:
N/A

>Fix:
>Release-Note:
>Audit-Trail:

From: Jason Evans <jasone@canonware.com>
To: "justin@skysoft.com.tw" <buggy@kkcity.com.tw>
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/31310: pthread broken with KVA_PAGES=512
Date: Tue, 16 Oct 2001 11:01:39 -0700

 On Tue, Oct 16, 2001 at 04:16:51PM +0800, justin@skysoft.com.tw wrote:
 > 
 > >Description:
 > KVA_PAGES in kernel config is set to 512, and I had 'make world'
 > to update /usr/lib.
 > 
 > When I run any program that use pthread (linked with libc_r), 
 > it prints:
 > 
 >  
 > Fatal error 'Cannot allocate red zone for initial thread' at line ? in file
 > /usr/src/lib/libc_r/uthread/uthread_init.c (errno = ?)
 > 
 > Abort trap (core dumped)
 >  
 > 
 > I didn't have this problem on 4.4-RC0 with ldscript.i386 and pmap.h 
 > patched as the following to enlarge kernel address space to 2G.
 
 I'm not sure I understand what you're doing, so let me try to summarize my
 understanding before guessing what the problem is.  It sounds like you are
 changing the amount of address space reserved for the kernel, which affects
 where the userland stack is mapped.  Look in
 src/lib/libc_r/uthread/uthread_init.c for the mmap() call that creates the
 red zone:
 
 		/*
 		 * Create a red zone below the main stack.  All other stacks are
 		 * constrained to a maximum size by the paramters passed to
 		 * mmap(), but this stack is only limited by resource limits, so
 		 * this stack needs an explicitly mapped red zone to protect the
 		 * thread stack that is just beyond.
 		 */
 		if (mmap((void *) USRSTACK - PTHREAD_STACK_INITIAL -
 		    PTHREAD_STACK_GUARD, PTHREAD_STACK_GUARD, 0, MAP_ANON,
 		    -1, 0) == MAP_FAILED)
 			PANIC("Cannot allocate red zone for initial thread");
 
 The code depends on the USRSTACK macro when determining where to create the
 red zone, and I suspect that its value is somewhere in kernel memory in
 your case.
 
 From src/sys/i386/include/vmparam.h:
 
 #define	VM_MAXUSER_ADDRESS	VADDR(UMAXPTDI, UMAXPTEOFF)
 
 #define	USRSTACK		VM_MAXUSER_ADDRESS
 
 Please let me know if after further diagnostic effort you discover this to
 be an actual problem in libc_r.
 
 Jason

From: Justin Chuang <justin@skysoft.com.tw>
To: freebsd-gnats-submit@FreeBSD.org
Cc: jasone@canonware.com
Subject: Re: kern/31310: pthread broken with KVA_PAGES=512
Date: Wed, 17 Oct 2001 15:18:25 +0800

 Thanks to Jason.
 
 I solved this problem by adding -DKVA_PAGES=512 in CFLAGS at etc/make.conf
 
 Libc_r include /usr/include/machine/pmap.h to determine 
 USRSTACK, but kernel configuration of KVA_PAGES is defined in
 /usr/src/sys/compile/XXXX/opt_global.h, so libc_r use wrong value of
 USRSTACK.
 
 Once after KVA_PAGES is documented in LINT, a note on this would be helpful
 for those who need pthread and have large memory.
 
 By the way, FreeBSD prior to 4.4-STABLE, including 4.4-RC0, seem to have
 wrong pmap.h that can't produce correct USRSTACK from KVA_PAGES macro. If a
 person want to upgrade to 4.4-STABLE and change KVA_PAGES, maybe he needs to
 make world twice to have correct pmap.h in /usr/include.

From: Bruce Evans <bde@zeta.org.au>
To: Jason Evans <jasone@canonware.com>
Cc: <freebsd-gnats-submit@FreeBSD.ORG>
Subject: Re: kern/31310: pthread broken with KVA_PAGES=512
Date: Wed, 17 Oct 2001 17:23:15 +1000 (EST)

 On Tue, 16 Oct 2001, Jason Evans wrote:
 
 >  I'm not sure I understand what you're doing, so let me try to summarize my
 >  understanding before guessing what the problem is.  It sounds like you are
 >  changing the amount of address space reserved for the kernel, which affects
 >  where the userland stack is mapped.  Look in
 >  src/lib/libc_r/uthread/uthread_init.c for the mmap() call that creates the
 >  red zone:
 >
 >  		/*
 >  		 * Create a red zone below the main stack.  All other stacks are
 >  		 * constrained to a maximum size by the paramters passed to
 >  		 * mmap(), but this stack is only limited by resource limits, so
 >  		 * this stack needs an explicitly mapped red zone to protect the
 >  		 * thread stack that is just beyond.
 >  		 */
 >  		if (mmap((void *) USRSTACK - PTHREAD_STACK_INITIAL -
 >  		    PTHREAD_STACK_GUARD, PTHREAD_STACK_GUARD, 0, MAP_ANON,
 >  		    -1, 0) == MAP_FAILED)
 >  			PANIC("Cannot allocate red zone for initial thread");
 >
 >  The code depends on the USRSTACK macro when determining where to create the
 >  red zone, and I suspect that its value is somewhere in kernel memory in
 >  your case.
 >
 >  From src/sys/i386/include/vmparam.h:
 >
 >  #define	VM_MAXUSER_ADDRESS	VADDR(UMAXPTDI, UMAXPTEOFF)
 >
 >  #define	USRSTACK		VM_MAXUSER_ADDRESS
 >
 >  Please let me know if after further diagnostic effort you discover this to
 >  be an actual problem in libc_r.
 
 Code like this (actually mainly bogus definitions of non-constants as
 constant in <sys/param.h>) is very broken.  USRSTACK depends on the
 kernel option KVA_PAGES.  Userland can't see kernel options headers,
 so USRSTACK should not be visible in userland.  But i386/include/param.h
 provides a bogus default for KVA_PAGES and consequently USRSTACK to
 move the breakage from compile time to run time.
 
 Some other bogus non-constant constants:
 - OPEN_MAX and CHILD_MAX.
 - PAGE_SIZE and pgtok() leaked out to userland a long time ago, desipite
   the existence of syscall and a sysctl to insulate them.  PAGE_SIZE is
   now an option on ia64's.
 - UPAGES.
 - MAXDSIZE, etc.  There are now tunables.  Anyone tuning them had better
   know all related magic addresses used deep in places like the above.
   There was some discussion of making them (variable) rlimits.  This stalled
   on the complications for knowing all the magic addresses and keeping them
   separate.
 
 Bruce
 
State-Changed-From-To: open->closed 
State-Changed-By: peter 
State-Changed-When: Mon Dec 17 14:58:12 PST 2001 
State-Changed-Why:  
Committed to -current on 2001/10/26, and -stable on 2001/11/03. 
(See src/lib/libc_r/uthread/uthread_init.c and uthread_stack.c etc) 

http://www.FreeBSD.org/cgi/query-pr.cgi?pr=31310 
>Unformatted:
