From nobody@FreeBSD.org  Fri Jan 17 00:17:20 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id E1A8FF44
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 17 Jan 2014 00:17:20 +0000 (UTC)
Received: from oldred.freebsd.org (oldred.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id CC5CB17A9
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 17 Jan 2014 00:17:20 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id s0H0HKj7038818
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 17 Jan 2014 00:17:20 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id s0H0HKOe038804;
	Fri, 17 Jan 2014 00:17:20 GMT
	(envelope-from nobody)
Message-Id: <201401170017.s0H0HKOe038804@oldred.freebsd.org>
Date: Fri, 17 Jan 2014 00:17:20 GMT
From: Alfred Perlstein <alfred@FreeBSD.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: bumping maxcpu causes witness to panic at boot
X-Send-Pr-Version: www-3.1
X-GNATS-Notify: alfred, tgs@norse-corp.com

>Number:         185831
>Category:       kern
>Synopsis:       [witness] [patch] bumping maxcpu causes witness to panic at boot
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    jhb
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Jan 17 00:20:00 UTC 2014
>Closed-Date:    
>Last-Modified:  Tue Apr 29 17:30:00 UTC 2014
>Originator:     Alfred Perlstein
>Release:        11
>Organization:
Norse Corporation
>Environment:
>Description:
If we bump maxcpu on our machine we get a panic at boot about running out of witness pending objects.



The following patch fixes it:
(https://github.com/alfredperlstein/freebsd/compare/witness_scaling)

diff --git a/sys/kern/subr_witness.c b/sys/kern/subr_witness.c
index 9315133..e1c8d1b 100644
--- a/sys/kern/subr_witness.c
+++ b/sys/kern/subr_witness.c
@@ -132,10 +132,19 @@
 /* Define this to check for blessed mutexes */
 #undef BLESSING
 
-#define	WITNESS_COUNT 		1024
-#define	WITNESS_CHILDCOUNT 	(WITNESS_COUNT * 4)
+/*
+ * Each cpu requires early locks so scale WITNESS_COUNT & WITNESS_PENDLIST
+ * based on MAXCPU
+ */
+#ifndef	WITNESS_COUNT
+#define	WITNESS_COUNT		(MAXCPU <= 64 ? 1024 : MAXCPU * 16)
+#endif
+
 #define	WITNESS_HASH_SIZE	251	/* Prime, gives load factor < 2 */
-#define	WITNESS_PENDLIST	1024
+
+#ifndef	WITNESS_PENDLIST
+#define	WITNESS_PENDLIST	(MAXCPU <= 64 ? 1024 : MAXCPU * 16)
+#endif
 
 /* Allocate 256 KB of stack data space */
 #define	WITNESS_LO_DATA_COUNT	2048
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->jhb 
Responsible-Changed-By: alfred 
Responsible-Changed-When: Fri Jan 17 00:41:11 UTC 2014 
Responsible-Changed-Why:  
John, can I get a review please? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=185831 

From: Peter Grehan <peterg@ptree32.com.au>
To: bug-followup@FreeBSD.org, alfred@FreeBSD.org
Cc:  
Subject: Re: kern/185831: [witness] [patch] bumping maxcpu causes witness
 to panic at boot
Date: Tue, 04 Mar 2014 11:28:26 -0800

 I have a fix for the WITNESS_PENDLIST. There is only one contributor to 
 the size of this that is dependent on MAXCPU. The others are static and 
 haven't changed in a long time, so the change is
 
 +#define        WITNESS_PENDLIST        (1024 + MAXCPU)
 -#define        WITNESS_PENDLIST        1024
 
 I booted an amd64 bhyve guest with a MAXCPU value of 256 and it didn't 
 hit the limit.
 
 The change has been OK'd by jhb@ so I will commit this shortly unless 
 anyone has objections.
 
 later,
 
 Peter.

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/185831: commit references a PR
Date: Tue, 29 Apr 2014 17:22:33 +0000 (UTC)

 Author: grehan
 Date: Tue Apr 29 17:22:29 2014
 New Revision: 265098
 URL: http://svnweb.freebsd.org/changeset/base/265098
 
 Log:
   Bump WITNESS_PENDLIST by MAXCPU to account for the
   pmap pvlist locks which are scaled by MAXCPU.
   
   This allows an amd64 system to boot with MAXCPU set
   to 256, which is currently FreeBSD's hard limit without
   x2apic support.
   
   Compile-tested for other arch's.
   
   PR:	185831
   Discussed with:		jhb
   MFC after:	3 weeks
 
 Modified:
   head/sys/kern/subr_witness.c
 
 Modified: head/sys/kern/subr_witness.c
 ==============================================================================
 --- head/sys/kern/subr_witness.c	Tue Apr 29 16:57:25 2014	(r265097)
 +++ head/sys/kern/subr_witness.c	Tue Apr 29 17:22:29 2014	(r265098)
 @@ -135,7 +135,7 @@ __FBSDID("$FreeBSD$");
  #define	WITNESS_COUNT 		1536
  #define	WITNESS_CHILDCOUNT 	(WITNESS_COUNT * 4)
  #define	WITNESS_HASH_SIZE	251	/* Prime, gives load factor < 2 */
 -#define	WITNESS_PENDLIST	1024
 +#define	WITNESS_PENDLIST	(1024 + MAXCPU)
  
  /* Allocate 256 KB of stack data space */
  #define	WITNESS_LO_DATA_COUNT	2048
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
