From tdb@buzzard.kent.ac.uk  Sat May 24 10:20:00 2014
Return-Path: <tdb@buzzard.kent.ac.uk>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id A9BA54AE
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 24 May 2014 10:20:00 +0000 (UTC)
Received: from mx1.kent.ac.uk (mx1.kent.ac.uk [129.12.21.39])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 6B69A2E71
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 24 May 2014 10:19:59 +0000 (UTC)
Received: from apophis.ukc.ac.uk ([129.12.4.11])
	by mx1.kent.ac.uk with esmtps (TLSv1:AES256-SHA:256)
	(Exim 4.72)
	(envelope-from <tdb@buzzard.kent.ac.uk>)
	id 1Wo93W-0006fU-Fd
	for FreeBSD-gnats-submit@freebsd.org; Sat, 24 May 2014 11:19:58 +0100
Received: from buzzard.kent.ac.uk ([129.12.3.201])
	by apophis.ukc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256)
	(Exim 4.80.1)
	(envelope-from <tdb@buzzard.kent.ac.uk>)
	id 1Wo93V-0001uv-Rc
	for FreeBSD-gnats-submit@freebsd.org; Sat, 24 May 2014 11:19:57 +0100
Received: from buzzard.kent.ac.uk (localhost [127.0.0.1])
	by buzzard.kent.ac.uk (8.14.9/8.14.9) with ESMTP id s4OAJu9i001380
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO)
	for <FreeBSD-gnats-submit@freebsd.org>; Sat, 24 May 2014 11:19:56 +0100 (BST)
	(envelope-from tdb@buzzard.kent.ac.uk)
Received: (from tdb@localhost)
	by buzzard.kent.ac.uk (8.14.9/8.14.9/Submit) id s4OAJuUA001379;
	Sat, 24 May 2014 11:19:56 +0100 (BST)
	(envelope-from tdb)
Message-Id: <201405241019.s4OAJuUA001379@buzzard.kent.ac.uk>
Date: Sat, 24 May 2014 11:19:56 +0100 (BST)
From: Tim Bishop <tim@bishnet.net>
Reply-To: Tim Bishop <tim@bishnet.net>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: Increase MAXCPU on amd64 systems
X-Send-Pr-Version: 3.114
X-GNATS-Notify:

>Number:         190169
>Category:       kern
>Synopsis:       [patch] Increase MAXCPU on amd64 systems
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Sat May 24 10:30:00 UTC 2014
>Closed-Date:    
>Last-Modified:  Sat May 24 17:03:31 UTC 2014
>Originator:     Tim Bishop
>Release:        FreeBSD 11.0-CURRENT amd64
>Organization:
>Environment:
System: FreeBSD buzzard 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r266603M: Sat May 24 03:25:35 BST 2014 tdb@buzzard:/usr/obj/usr/src/sys/GENERIC amd64

	4x Intel(R) Xeon(R) CPU E7-4830 v2 @ 2.20GHz (2200.05-MHz K8-class CPU)
	This gives 40 cores with a total of 80 threads.

>Description:
	In param.h MAXCPU is set to 64. This results in 16 of the
	80 threads being disabled at boot time, and errors like:

	FreeBSD/SMP: Multiprocessor System Detected: 64 CPUs
	FreeBSD/SMP: 3 package(s) x 10 core(s) x 2 SMT threads
	 cpu0 (BSP): APIC ID:  0
	 cpu1 (AP): APIC ID:  1
	 ...
	 cpu62 (AP): APIC ID: 98
	 cpu63 (AP): APIC ID: 99
	  cpu (AP): APIC ID: 100 (disabled)
	  cpu (AP): APIC ID: 101 (disabled)
	  cpu (AP): APIC ID: 102 (disabled)
	  cpu (AP): APIC ID: 103 (disabled)
	  cpu (AP): APIC ID: 104 (disabled)
	  cpu (AP): APIC ID: 105 (disabled)
	  cpu (AP): APIC ID: 112 (disabled)
	  cpu (AP): APIC ID: 113 (disabled)
	  cpu (AP): APIC ID: 114 (disabled)
	  cpu (AP): APIC ID: 115 (disabled)
	  cpu (AP): APIC ID: 116 (disabled)
	  cpu (AP): APIC ID: 117 (disabled)
	  cpu (AP): APIC ID: 118 (disabled)
	  cpu (AP): APIC ID: 119 (disabled)
	  cpu (AP): APIC ID: 120 (disabled)
	  cpu (AP): APIC ID: 121 (disabled)
	WARNING: Non-uniform processors.
	WARNING: Using suboptimal topology.

>How-To-Repeat:
	Boot any recent FreeBSD version, including HEAD, on a system
	with more than 64 CPU cores/threads.

>Fix:
	The current top end processors have 15 cores and 30 threads.
	So a quad socket system would have a total of 120 threads.
	I believe this should be supported in GENERIC so FreeBSD
	works out of the box. The patch below does this.

	It'd be nice to see this MFCed to stable/10 so it makes the
	10.1 release. It'd be even nicer to see it MFCed to stable/9
	for 9.3, but I suspect it'll be too late for that.

	For what it's worth I tested the most recent Ubuntu release
	and it detected all the threads out of the box.

	Some consideration should also be given to kern/185831. It
	has been committed but I don't think it has been MFCed yet.
	I can confirm that I don't receive any panic on HEAD with
	MAXCPU set to 128 and with witness enabled.

--- diff begins here ---
Index: sys/amd64/include/param.h
===================================================================
--- sys/amd64/include/param.h	(revision 266603)
+++ sys/amd64/include/param.h	(working copy)
@@ -65,7 +65,7 @@
 
 #if defined(SMP) || defined(KLD_MODULE)
 #ifndef MAXCPU
-#define MAXCPU		64
+#define MAXCPU		128
 #endif
 #else
 #define MAXCPU		1
--- diff ends here ---


>Release-Note:
>Audit-Trail:

From: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: kern/190169: Increase MAXCPU on amd64 systems
Date: Sat, 24 May 2014 10:53:37 +0000

 When I had checked about a year ago it was possible to buy >> 180 thread =
 machines off the shelf already;  I=92d argue for 256.=

From: "Ranjan1018 ." <214748mv@gmail.com>
To: bug-followup@FreeBSD.org, tim@bishnet.net
Cc:  
Subject: Re: kern/190169: Increase MAXCPU on amd64 systems
Date: Sat, 24 May 2014 14:32:17 +0200

 --047d7b339db15e61c204fa2489a8
 Content-Type: text/plain; charset=UTF-8
 
 This motherboard needs 180 threads, if I am not wrong:
 http://www.supermicro.com/products/motherboard/Xeon7000/7500/X8OBN-F.cfm
 
 --047d7b339db15e61c204fa2489a8--

From: Tim Bishop <tim@bishnet.net>
To: "Ranjan1018 ." <214748mv@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/190169: Increase MAXCPU on amd64 systems
Date: Sat, 24 May 2014 13:42:04 +0100

 On Sat, May 24, 2014 at 02:32:17PM +0200, Ranjan1018 . wrote:
 > This motherboard needs 180 threads, if I am not wrong:
 > http://www.supermicro.com/products/motherboard/Xeon7000/7500/X8OBN-F.cfm
 
 I'd certainly have no objections to it being set even higher than I
 suggested :-)
 
 According to kern/185831 there's a hard limit of 256 at the moment, so
 it could just be set to that.
 
 Tim.
 
 -- 
 Tim Bishop
 http://www.bishnet.net/tim/
 PGP Key: 0x6C226B37FDF38D55
 
>Unformatted:
