From nobody@FreeBSD.org  Mon Feb 11 02:51:48 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 38AD516A420
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 11 Feb 2008 02:51:48 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 231BD13C4E5
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 11 Feb 2008 02:51:48 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m1B2nZFI015877
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 11 Feb 2008 02:49:35 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id m1B2nZRQ015876;
	Mon, 11 Feb 2008 02:49:35 GMT
	(envelope-from nobody)
Message-Id: <200802110249.m1B2nZRQ015876@www.freebsd.org>
Date: Mon, 11 Feb 2008 02:49:35 GMT
From: Arthur Hartwig <arthur.hartwig@nokia.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Intel "G0" stepping Xeon CPUs perform badly
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         120516
>Category:       i386
>Synopsis:       [i386] Intel "G0" stepping Xeon CPUs perform badly
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    jhb
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Feb 11 03:00:04 UTC 2008
>Closed-Date:    Wed Mar 19 16:39:56 UTC 2008
>Last-Modified:  Wed Mar 19 16:39:56 UTC 2008
>Originator:     Arthur Hartwig
>Release:        6.3 but applies to all FreeBSD versions
>Organization:
Nokia
>Environment:
>Description:
The Intel Xeon "G0" stepping CPU (family/id values = 0x6fb) perform badly.

Intel has released some documentation stating that the MTRR base and
mask register values for the CPU stepping should hold values significant
to 38 bits corresponding to the 38 bits of physical address on these CPUs
(up from 36 bits of physical address on previous CPUs).

FreeBSD leaves the boot CPU MTRRs alone but (in i686_mrstoreone() in
sys/i386/i386/i686_mem.c) masks the AP MTRRs to 36 bits. In Nokia
testing using a custome memory stress tester we saw a "B3" stepping
CPU (family/id value of 0x6f7) do around 38 test cycles per minute on
each of the two CPUs enabled. When the two quad core CPU packages were
replaced by G0 stepping quad core CPUs the number of test cycles dropped
to 32 on CPU0 and 0.6 on CPU1. Though the BIOS was correctly setting
the MTRRs to the 38 bit values recommended by Intel FreeBSD was masking
them to 36 bits on the non boot cpu (e.g. base and mask of 0x00000000c0000000 and 0x00000003f0000000 became 0x00000000c0000000 and 0x00000000f0000000).


Because the BIOS was known to correctly set the MTRRs the call to
mem_range_AP_init() in init_secondary() in i386/i386/mp_machdep.c was
removed so the AP MTRRs were not modified. Then CPU0 ran about 38 test
cycles per minute and CPU1 about 39 test cycles per minute.


>How-To-Repeat:

>Fix:
If it is felt that FreeBSD cannot rely on the BIOS to correctly set the
MTRRs then the best fix is probably to modify i686_mrstoreone() to use
the cpuid instruction to determine the physical address width of memory
and construct appropriately sized masks.

This problem most likely also affects the amd64 architecture but I have
not looked into that issue.

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-i386 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Feb 11 07:08:20 UTC 2008 
Responsible-Changed-Why:  
Reclassify. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120516 
Responsible-Changed-From-To: freebsd-i386->jhb 
Responsible-Changed-By: kris 
Responsible-Changed-When: Mon Feb 25 19:44:35 UTC 2008 
Responsible-Changed-Why:  
John plans to work on this 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120516 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: i386/120516: commit references a PR
Date: Wed, 12 Mar 2008 22:09:31 +0000 (UTC)

 jhb         2008-03-12 22:09:19 UTC
 
   FreeBSD src repository
 
   Modified files:
     sys/amd64/amd64      amd64_mem.c 
     sys/amd64/include    specialreg.h 
     sys/i386/i386        i686_mem.c 
     sys/i386/include     specialreg.h 
   Log:
   The variable MTRR registers actually have variable-sized PhysBase and
   PhysMask fields based on the number of physical address bits supported
   by the current CPU.  The old code assumed 36 bits on i386 and 40 bits on
   amd64.  In truth, all Intel CPUs up until recently used 36 bits (a newer
   Intel CPU uses 38 bits) and all the Opteron CPUs used 40 bits.
   
   In at least one case (the new Intel CPU) having the size of the mask field
   wrong resulted in writing questionable values into the MTRR registers on
   the application processors (BSP as well if you modify the MTRRs via
   memcontrol or running X, etc.).  The result of the questionable physmask
   was that all of memory was apparently treated as uncached rather than
   write-back resulting in a very significant performance hit.
   
   Fix this by constructing a run-time mask for the PhysBase and PhysMask
   fields based on the number of physical address bits supported by the CPU.
   All 64-bit capable CPUs provide a count of PA bits supported via the
   0x80000008 extended CPUID feature, so use that if it is available.  If that
   feature is not available, then assume 36 PA bits.
   
   While I'm here, expand the (now-unused) macros for the PhysBase and
   PhysMask fields to the current largest possible value (52 PA bits).
   
   MFC after:      1 week
   PR:             i386/120516
   Reported by:    Nokia
   
   Revision  Changes    Path
   1.29      +24 -6     src/sys/amd64/amd64/amd64_mem.c
   1.45      +2 -2      src/sys/amd64/include/specialreg.h
   1.28      +24 -6     src/sys/i386/i386/i686_mem.c
   1.44      +2 -2      src/sys/i386/include/specialreg.h
 _______________________________________________
 cvs-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/cvs-all
 To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
 

From: John Baldwin <jhb@FreeBSD.org>
To: bug-followup@FreeBSD.org, arthur.hartwig@nokia.com
Cc:  
Subject: Re: i386/120516: [i386] Intel "G0" stepping Xeon CPUs perform badly
Date: Wed, 12 Mar 2008 18:11:06 -0400

 I've committed a fix to HEAD.  I will come up with a patch for 6.x/7.x for 
 testing tomorrow.  You can test it by just grabbing 
 sys/amd64/amd64/amd64_mem.c, sys/amd64/include/specialreg.h, 
 sys/i386/i386/i686_mem.c, and sys/i386/include/specialreg.h from HEAD and the 
 M_MEMDESC change to sys/amd64/amd64/mem.c.
 
 -- 
 John Baldwin
State-Changed-From-To: open->closed 
State-Changed-By: jhb 
State-Changed-When: Wed Mar 19 16:39:32 UTC 2008 
State-Changed-Why:  
Patch MFC'd to RELENG_6 and RELENG_7. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120516 
>Unformatted:
