From nobody@FreeBSD.org  Thu Jan 23 23:09:04 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id 37040AB0
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 23 Jan 2014 23:09:04 +0000 (UTC)
Received: from oldred.freebsd.org (oldred.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 02C571720
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 23 Jan 2014 23:09:04 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
	by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id s0NN93FS080403
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 23 Jan 2014 23:09:03 GMT
	(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
	by oldred.freebsd.org (8.14.5/8.14.5/Submit) id s0NN93AB080397;
	Thu, 23 Jan 2014 23:09:03 GMT
	(envelope-from nobody)
Message-Id: <201401232309.s0NN93AB080397@oldred.freebsd.org>
Date: Thu, 23 Jan 2014 23:09:03 GMT
From: Simon Matter <simon.matter@invoca.ch>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD 10 crashes as KVM guest on GNU/Linux on AMD family 10h CPUs
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         186061
>Category:       amd64
>Synopsis:       [kvm] FreeBSD 10 crashes as KVM guest on GNU/Linux on AMD family 10h CPUs
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    alc
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jan 23 23:10:01 UTC 2014
>Closed-Date:    
>Last-Modified:  Wed May  7 16:30:00 UTC 2014
>Originator:     Simon Matter
>Release:        10.0-RELEASE
>Organization:
Invoca Systems
>Environment:
FreeBSD install.corp.invoca.ch 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014     root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
I've installed FreeBSD 10.0-RELEASE as KVM guest on a CentOS 6.5 system. It all works well but as soon as some load hits the FreeBSD guest, the virtual instance just reboots.
>How-To-Repeat:
Install CentOS 6.5 x86_64 on a HP Microserver (or any other AMD family 10h box like a DL585 G7).
Create a KVM guest and install FreeBSD 10 on it.
Start some applications on the guest and wait a moment, it will soon boot again.

Note: 
1) There is no panic on the guest, it just gets virtually switched off and on again.
2) Both x86 archs are affected, i386 and amd64 both crash the same way.
>Fix:
After the guest has rebooted, dmesg on the host shows this nice message:

KVM: Guest triggered AMD Erratum 383

Looking at the FreeBSD kernel code it's clear that the kernel usually detects and deals with AMD Erratum 383 nicely. However, in the situation as a KVM guest, this doesn't work anymore. To make migration of guest between different CPU families possible, Qemu-KVM reports a different family to the guest than the real CPU has. The effect is that the kernels auto detection simply doesn't work anymore.

The first thing that came to mind is why not make workaround_erratum383 tunable via sysctl?

But maybe it's better to improve auto detection? I found three places where auto detection takes place:

x86/x86/mca.c
in mca_setup()
  723         if (cpu_vendor_id == CPU_VENDOR_AMD &&
  724             CPUID_TO_FAMILY(cpu_id) == 0x10 && amd10h_L1TP)
  725                 workaround_erratum383 = 1;

i386/i386/pmap.c
in pmap_init()
  759         if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
  760             CPUID_TO_FAMILY(cpu_id) == 0x10)
  761                 workaround_erratum383 = 1;

amd64/amd64/pmap.c
in pmap_init()
 1012         if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 1013             CPUID_TO_FAMILY(cpu_id) == 0x10)
 1014                 workaround_erratum383 = 1;

Wouldn't it make sense to change these so that workaround_erratum383 == 1 if

a) cpu_vendor_id == CPU_VENDOR_AMD && CPUID_TO_FAMILY(cpu_id) == 0x10)

or

b) vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD

is true?

I don't understand mca_setup() nor pmap_init() but it seems workaround_erratum383 should always be enabled if
a) CPU vendor is AMD and CPU family is 10h
or
b) we are a guest VM and CPU vendor is AMD (no matter which family)

Thanks for looking into it!

Regards,
Simon

>Release-Note:
>Audit-Trail:

From: "Simon Matter" <simon.matter@invoca.ch>
To: bug-followup@FreeBSD.org
Cc: simon.matter@invoca.ch
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
 AMD family 10h CPUs
Date: Thu, 6 Feb 2014 11:46:41 +0100

 ------=_20140206114641_95473
 Content-Type: text/plain; charset="iso-8859-1"
 Content-Transfer-Encoding: 8bit
 
 Hi,
 
 After thinking about it again it seems the proposed solution may not be
 enough. At least KVM allows to migrate guests from an Intel to an AMD
 processor. That means in case of running as a vm guest, it's required to
 always enable "AMD Erratum 383" workaround. Otherwise, after migration to
 an affected AMD Family 10h processor, the guest could triggered AMD
 Erratum 383.
 
 I've tried to implement this and attached patch fixes the problem for me.
 Would me nice if someone with more experience than me could have a look at
 it.
 
 Thanks,
 Simon
 ------=_20140206114641_95473
 Content-Type: text/x-diff; name="vm-erratum383.patch"
 Content-Transfer-Encoding: 8bit
 Content-Disposition: attachment; filename="vm-erratum383.patch"
 
 --- /usr/src/sys/x86/x86/mca.c.orig	2014-01-16 21:35:03.000000000 +0100
 +++ /usr/src/sys/x86/x86/mca.c	2014-02-05 22:15:53.109619475 +0100
 @@ -720,8 +720,8 @@
  	 * parity (L1TP) errors is disabled, enable the recommended workaround
  	 * for Erratum 383.
  	 */
 -	if (cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10 && amd10h_L1TP)
 +	if (vm_guest != VM_GUEST_NO || (cpu_vendor_id == CPU_VENDOR_AMD &&
 +	    CPUID_TO_FAMILY(cpu_id) == 0x10 && amd10h_L1TP))
  		workaround_erratum383 = 1;
  
  	mca_banks = mcg_cap & MCG_CAP_COUNT;
 --- /usr/src/sys/i386/i386/pmap.c.orig	2014-01-16 21:33:36.000000000 +0100
 +++ /usr/src/sys/i386/i386/pmap.c	2014-02-05 22:25:28.395821316 +0100
 @@ -752,12 +752,12 @@
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 +	 * If the kernel is running in a virtual machine on any processor
 +	 * family, then it must assume that MCA is enabled by the virtual
 +	 * machine monitor and the vm may migrate to an AMD Family 10h
 +	 * processor.
  	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	if (vm_guest != VM_GUEST_NO)
  		workaround_erratum383 = 1;
  
  	/*
 --- /usr/src/sys/amd64/amd64/pmap.c.orig	2014-01-16 21:33:04.000000000 +0100
 +++ /usr/src/sys/amd64/amd64/pmap.c	2014-02-05 22:28:25.814349113 +0100
 @@ -1005,12 +1005,12 @@
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 +	 * If the kernel is running in a virtual machine on any processor
 +	 * family, then it must assume that MCA is enabled by the virtual
 +	 * machine monitor and the vm may migrate to an AMD Family 10h
 +	 * processor.
  	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	if (vm_guest != VM_GUEST_NO)
  		workaround_erratum383 = 1;
  
  	/*
 ------=_20140206114641_95473--
 
 

From: "Simon Matter" <simon.matter@invoca.ch>
To: bug-followup@FreeBSD.org
Cc: simon.matter@invoca.ch
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
 AMD family 10h CPUs
Date: Wed, 12 Feb 2014 08:30:51 +0100

 ------=_20140212083051_97180
 Content-Type: text/plain; charset="iso-8859-1"
 Content-Transfer-Encoding: 8bit
 
 As noted by John Baldwin the change to mca.c is not needed. Attached patch
 is what I'm using now with success.
 
 BTW: setting vm.pmap.pg_ps_enabled="0" in loader.conf does also mitigate
 the issue but I guess it's not the optimal solution.
 
 Regards,
 Simon
 ------=_20140212083051_97180
 Content-Type: text/x-diff; name="vm-erratum383.patch"
 Content-Transfer-Encoding: 8bit
 Content-Disposition: attachment; filename="vm-erratum383.patch"
 
 --- /usr/src/sys/i386/i386/pmap.c.orig	2014-01-16 21:33:36.000000000 +0100
 +++ /usr/src/sys/i386/i386/pmap.c	2014-02-05 22:25:28.395821316 +0100
 @@ -752,12 +752,12 @@
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 +	 * If the kernel is running in a virtual machine on any processor
 +	 * family, then it must assume that MCA is enabled by the virtual
 +	 * machine monitor and the vm may migrate to an AMD Family 10h
 +	 * processor.
  	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	if (vm_guest != VM_GUEST_NO)
  		workaround_erratum383 = 1;
  
  	/*
 --- /usr/src/sys/amd64/amd64/pmap.c.orig	2014-01-16 21:33:04.000000000 +0100
 +++ /usr/src/sys/amd64/amd64/pmap.c	2014-02-05 22:28:25.814349113 +0100
 @@ -1005,12 +1005,12 @@
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 +	 * If the kernel is running in a virtual machine on any processor
 +	 * family, then it must assume that MCA is enabled by the virtual
 +	 * machine monitor and the vm may migrate to an AMD Family 10h
 +	 * processor.
  	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	if (vm_guest != VM_GUEST_NO)
  		workaround_erratum383 = 1;
  
  	/*
 ------=_20140212083051_97180--
 
 

From: Alan Cox <alc@rice.edu>
To: bug-followup@FreeBSD.org, simon.matter@invoca.ch
Cc:  
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
 AMD family 10h CPUs
Date: Sat, 15 Feb 2014 12:09:58 -0600

 This is a multi-part message in MIME format.
 --------------080509080501070706030501
 Content-Type: text/plain; charset=ISO-8859-1
 Content-Transfer-Encoding: 7bit
 
 Can you please verify that the attached patch addresses the problem for
 you?  Aside from addressing the crash, the objective of this patch is
 avoid enabling the workaround for perpetuity on all past Intel and
 future AMD/Intel cores on account of one broken AMD core.  The systems
 that I've seen for VM migration look at the reported processor feature
 sets and only migrate among machines with like feature sets.  So, if the
 feature set includes at least one feature that is not supported by AMD
 Family 10h or earlier AMD cores, then we shouldn't need to enable the
 workaround.
 
 
 --------------080509080501070706030501
 Content-Type: text/plain; charset=ISO-8859-15;
  name="PR186061v2.patch"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
  filename="PR186061v2.patch"
 
 Index: amd64/amd64/pmap.c
 ===================================================================
 --- amd64/amd64/pmap.c	(revision 261162)
 +++ amd64/amd64/pmap.c	(working copy)
 @@ -1008,12 +1008,18 @@ pmap_init(void)
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
  	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 Index: i386/i386/pmap.c
 ===================================================================
 --- i386/i386/pmap.c	(revision 261162)
 +++ i386/i386/pmap.c	(working copy)
 @@ -752,12 +752,18 @@ pmap_init(void)
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
  	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 
 --------------080509080501070706030501--
Responsible-Changed-From-To: freebsd-amd64->alc 
Responsible-Changed-By: alc 
Responsible-Changed-When: Sat Feb 15 18:24:18 UTC 2014 
Responsible-Changed-Why:  
Take over this PR. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=186061 

From: "Simon Matter" <simon.matter@invoca.ch>
To: "Alan Cox" <alc@rice.edu>
Cc: bug-followup@freebsd.org
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
 AMD family 10h CPUs
Date: Tue, 18 Feb 2014 00:25:57 +0100

 > Can you please verify that the attached patch addresses the problem for
 > you?  Aside from addressing the crash, the objective of this patch is
 > avoid enabling the workaround for perpetuity on all past Intel and
 > future AMD/Intel cores on account of one broken AMD core.  The systems
 > that I've seen for VM migration look at the reported processor feature
 > sets and only migrate among machines with like feature sets.  So, if the
 > feature set includes at least one feature that is not supported by AMD
 > Family 10h or earlier AMD cores, then we shouldn't need to enable the
 > workaround.
 
 I can confirm that the patch works on my test system by enabling the AMD
 erratum 383 workaround. But, AFAIK it means that the workaround will be
 enabled for most KVM setups. That's because, by default, KVM uses a low
 CPU feature set and a low cpu family (6) to be able to migrate a VM in
 almost every case. Therefore it would make sense to have a sysctl option
 to override the auto detected setting so that those who know what they are
 doing can force the state of the erratum 383 workaround.
 
 Regards,
 Simon
 

From: "Simon Matter" <simon.matter@invoca.ch>
To: "Alan Cox" <alc@rice.edu>
Cc: bug-followup@freebsd.org
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
 AMD family 10h CPUs
Date: Tue, 18 Feb 2014 14:33:32 +0100

 > Can you please verify that the attached patch addresses the problem for
 > you?  Aside from addressing the crash, the objective of this patch is
 > avoid enabling the workaround for perpetuity on all past Intel and
 > future AMD/Intel cores on account of one broken AMD core.  The systems
 > that I've seen for VM migration look at the reported processor feature
 > sets and only migrate among machines with like feature sets.  So, if the
 > feature set includes at least one feature that is not supported by AMD
 > Family 10h or earlier AMD cores, then we shouldn't need to enable the
 > workaround.
 
 What about keeping the whole CPU detection as it is and just make
 erratum383 a tunable sysctl so that those who run an affected
 configuration can enable it? I've tried it like below and put
 hw.mca.erratum383="1" into loader.conf.
 
 --- sys/x86/x86/mca.c.orig    2014-01-16 21:35:03.000000000 +0100
 +++ sys/x86/x86/mca.c 2014-02-18 11:34:07.189148894 +0100
 @@ -100,9 +100,10 @@
  SYSCTL_INT(_hw_mca, OID_AUTO, amd10h_L1TP, CTLFLAG_RDTUN, &amd10h_L1TP, 0,
      "Administrative toggle for logging of level one TLB parity (L1TP)
 errors");
 
 -int workaround_erratum383;
 -SYSCTL_INT(_hw_mca, OID_AUTO, erratum383, CTLFLAG_RD,
 &workaround_erratum383, 0,
 -    "Is the workaround for Erratum 383 on AMD Family 10h processors
 enabled?");
 +int workaround_erratum383 = 0;
 +TUNABLE_INT("hw.mca.erratum383", &workaround_erratum383);
 +SYSCTL_INT(_hw_mca, OID_AUTO, erratum383, CTLFLAG_RDTUN,
 &workaround_erratum383, 0,
 +    "Administrative toggle for enabling workaround for Erratum 383 on AMD
 Family 10h processors");
 
  static STAILQ_HEAD(, mca_internal) mca_freelist;
  static int mca_freecount;
 
 

From: Alan Cox <alc@rice.edu>
To: Simon Matter <simon.matter@invoca.ch>
Cc: bug-followup@freebsd.org
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
  AMD family 10h CPUs
Date: Wed, 19 Feb 2014 12:32:09 -0600

 On 02/18/2014 07:33, Simon Matter wrote:
 >> Can you please verify that the attached patch addresses the problem for
 >> you?  Aside from addressing the crash, the objective of this patch is
 >> avoid enabling the workaround for perpetuity on all past Intel and
 >> future AMD/Intel cores on account of one broken AMD core.  The systems
 >> that I've seen for VM migration look at the reported processor feature
 >> sets and only migrate among machines with like feature sets.  So, if the
 >> feature set includes at least one feature that is not supported by AMD
 >> Family 10h or earlier AMD cores, then we shouldn't need to enable the
 >> workaround.
 > What about keeping the whole CPU detection as it is and just make
 > erratum383 a tunable sysctl so that those who run an affected
 > configuration can enable it? I've tried it like below and put
 > hw.mca.erratum383="1" into loader.conf.
 
 I would prefer to do the opposite, that is, automatically enable the
 workaround if the kernel can't be prove that the underlying processor is
 unaffected and provide a tunable for blocking the activation of the
 workaround in pmap.c.
 
 > --- sys/x86/x86/mca.c.orig    2014-01-16 21:35:03.000000000 +0100
 > +++ sys/x86/x86/mca.c 2014-02-18 11:34:07.189148894 +0100
 > @@ -100,9 +100,10 @@
 >  SYSCTL_INT(_hw_mca, OID_AUTO, amd10h_L1TP, CTLFLAG_RDTUN, &amd10h_L1TP, 0,
 >      "Administrative toggle for logging of level one TLB parity (L1TP)
 > errors");
 >
 > -int workaround_erratum383;
 > -SYSCTL_INT(_hw_mca, OID_AUTO, erratum383, CTLFLAG_RD,
 > &workaround_erratum383, 0,
 > -    "Is the workaround for Erratum 383 on AMD Family 10h processors
 > enabled?");
 > +int workaround_erratum383 = 0;
 > +TUNABLE_INT("hw.mca.erratum383", &workaround_erratum383);
 > +SYSCTL_INT(_hw_mca, OID_AUTO, erratum383, CTLFLAG_RDTUN,
 > &workaround_erratum383, 0,
 > +    "Administrative toggle for enabling workaround for Erratum 383 on AMD
 > Family 10h processors");
 >
 >  static STAILQ_HEAD(, mca_internal) mca_freelist;
 >  static int mca_freecount;
 >
 >
 >
 

From: Alan Cox <alc@rice.edu>
To: Simon Matter <simon.matter@invoca.ch>
Cc: bug-followup@freebsd.org
Subject: Re: amd64/186061: FreeBSD 10 crashes as KVM guest on GNU/Linux on
  AMD family 10h CPUs
Date: Thu, 20 Feb 2014 12:17:51 -0600

 On 02/17/2014 17:25, Simon Matter wrote:
 >> Can you please verify that the attached patch addresses the problem for
 >> you?  Aside from addressing the crash, the objective of this patch is
 >> avoid enabling the workaround for perpetuity on all past Intel and
 >> future AMD/Intel cores on account of one broken AMD core.  The systems
 >> that I've seen for VM migration look at the reported processor feature
 >> sets and only migrate among machines with like feature sets.  So, if the
 >> feature set includes at least one feature that is not supported by AMD
 >> Family 10h or earlier AMD cores, then we shouldn't need to enable the
 >> workaround.
 > I can confirm that the patch works on my test system by enabling the AMD
 > erratum 383 workaround. But, AFAIK it means that the workaround will be
 > enabled for most KVM setups. That's because, by default, KVM uses a low
 > CPU feature set and a low cpu family (6) to be able to migrate a VM in
 > almost every case. Therefore it would make sense to have a sysctl option
 > to override the auto detected setting so that those who know what they are
 > doing can force the state of the erratum 383 workaround.
 >
 
 If you're restricted to Family 6 features, then you're losing some
 features that will have more impact on performance than the overhead of
 this workaround.  For example, in just the pmap, you're losing support
 for 1 GB pages, which are used to implement the direct map, and you're
 losing the population count instruction, which sees moderate use on one
 of the pmap data structures.  So, I don't think it's worth expending too
 much effort on avoiding the workaround on older processors.
 
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/186061: commit references a PR
Date: Sat, 22 Feb 2014 18:53:51 +0000 (UTC)

 Author: alc
 Date: Sat Feb 22 18:53:42 2014
 New Revision: 262338
 URL: http://svnweb.freebsd.org/changeset/base/262338
 
 Log:
   When the kernel is running in a virtual machine, it cannot rely upon the
   processor family to determine if the workaround for AMD Family 10h Erratum
   383 should be enabled.  To enable virtual machine migration among a
   heterogeneous collection of physical machines, the hypervisor may have
   been configured to report an older processor family with a reduced feature
   set.  Effectively, the reported processor family and its features are like
   a "least common denominator" for the collection of machines.
   
   Therefore, when the kernel is running in a virtual machine, instead of
   relying upon the processor family, we now test for features that prove
   that the underlying processor is not affected by the erratum.  (The
   features that we test for are unlikely to ever be emulated in software
   on an affected physical processor.)
   
   PR:		186061
   Tested by:	Simon Matter
   Discussed with:	jhb, neel
   MFC after:	2 weeks
 
 Modified:
   head/sys/amd64/amd64/pmap.c
   head/sys/i386/i386/pmap.c
 
 Modified: head/sys/amd64/amd64/pmap.c
 ==============================================================================
 --- head/sys/amd64/amd64/pmap.c	Sat Feb 22 17:51:10 2014	(r262337)
 +++ head/sys/amd64/amd64/pmap.c	Sat Feb 22 18:53:42 2014	(r262338)
 @@ -1005,12 +1005,18 @@ pmap_init(void)
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 
 Modified: head/sys/i386/i386/pmap.c
 ==============================================================================
 --- head/sys/i386/i386/pmap.c	Sat Feb 22 17:51:10 2014	(r262337)
 +++ head/sys/i386/i386/pmap.c	Sat Feb 22 18:53:42 2014	(r262338)
 @@ -750,12 +750,18 @@ pmap_init(void)
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: Oliver Brandmueller <ob@e-Gitt.NET>
To: bug-followup@FreeBSD.org, simon.matter@invoca.ch
Cc:  
Subject: Re: amd64/186061: [kvm] FreeBSD 10 crashes as KVM guest on GNU/Linux
 on AMD family 10h CPUs
Date: Tue, 6 May 2014 11:43:09 +0200

 Any reason, this did not get MFC'd after 2 weeks? I'd really appreciate 
 seeing the fix in 10-STABLE, maybe even as bugfix before 10.1-R via 
 freebsd-update

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/186061: commit references a PR
Date: Wed,  7 May 2014 00:32:54 +0000 (UTC)

 Author: alc
 Date: Wed May  7 00:32:49 2014
 New Revision: 265476
 URL: http://svnweb.freebsd.org/changeset/base/265476
 
 Log:
   MFC r262338
     When the kernel is running in a virtual machine, it cannot rely upon the
     processor family to determine if the workaround for AMD Family 10h Erratum
     383 should be enabled.  To enable virtual machine migration among a
     heterogeneous collection of physical machines, the hypervisor may have
     been configured to report an older processor family with a reduced feature
     set.  Effectively, the reported processor family and its features are like
     a "least common denominator" for the collection of machines.
   
     Therefore, when the kernel is running in a virtual machine, instead of
     relying upon the processor family, we now test for features that prove
     that the underlying processor is not affected by the erratum.  (The
     features that we test for are unlikely to ever be emulated in software
     on an affected physical processor.)
   
   PR:		186061
 
 Modified:
   stable/10/sys/amd64/amd64/pmap.c
   stable/10/sys/i386/i386/pmap.c
 Directory Properties:
   stable/10/   (props changed)
 
 Modified: stable/10/sys/amd64/amd64/pmap.c
 ==============================================================================
 --- stable/10/sys/amd64/amd64/pmap.c	Tue May  6 23:28:48 2014	(r265475)
 +++ stable/10/sys/amd64/amd64/pmap.c	Wed May  7 00:32:49 2014	(r265476)
 @@ -1008,12 +1008,18 @@ pmap_init(void)
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 
 Modified: stable/10/sys/i386/i386/pmap.c
 ==============================================================================
 --- stable/10/sys/i386/i386/pmap.c	Tue May  6 23:28:48 2014	(r265475)
 +++ stable/10/sys/i386/i386/pmap.c	Wed May  7 00:32:49 2014	(r265476)
 @@ -752,12 +752,18 @@ pmap_init(void)
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/186061: commit references a PR
Date: Wed,  7 May 2014 15:52:45 +0000 (UTC)

 Author: alc
 Date: Wed May  7 15:52:41 2014
 New Revision: 265554
 URL: http://svnweb.freebsd.org/changeset/base/265554
 
 Log:
   MFC r262338
     When the kernel is running in a virtual machine, it cannot rely upon the
     processor family to determine if the workaround for AMD Family 10h Erratum
     383 should be enabled.  To enable virtual machine migration among a
     heterogeneous collection of physical machines, the hypervisor may have
     been configured to report an older processor family with a reduced feature
     set.  Effectively, the reported processor family and its features are like
     a "least common denominator" for the collection of machines.
   
     Therefore, when the kernel is running in a virtual machine, instead of
     relying upon the processor family, we now test for features that prove
     that the underlying processor is not affected by the erratum.  (The
     features that we test for are unlikely to ever be emulated in software
     on an affected physical processor.)
   
   PR:		186061
 
 Modified:
   stable/9/sys/amd64/amd64/pmap.c
   stable/9/sys/i386/i386/pmap.c
 Directory Properties:
   stable/9/sys/   (props changed)
 
 Modified: stable/9/sys/amd64/amd64/pmap.c
 ==============================================================================
 --- stable/9/sys/amd64/amd64/pmap.c	Wed May  7 15:34:04 2014	(r265553)
 +++ stable/9/sys/amd64/amd64/pmap.c	Wed May  7 15:52:41 2014	(r265554)
 @@ -797,12 +797,18 @@ pmap_init(void)
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 
 Modified: stable/9/sys/i386/i386/pmap.c
 ==============================================================================
 --- stable/9/sys/i386/i386/pmap.c	Wed May  7 15:34:04 2014	(r265553)
 +++ stable/9/sys/i386/i386/pmap.c	Wed May  7 15:52:41 2014	(r265554)
 @@ -768,12 +768,18 @@ pmap_init(void)
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/186061: commit references a PR
Date: Wed,  7 May 2014 16:28:40 +0000 (UTC)

 Author: alc
 Date: Wed May  7 16:28:36 2014
 New Revision: 265556
 URL: http://svnweb.freebsd.org/changeset/base/265556
 
 Log:
   MFC r262338
     When the kernel is running in a virtual machine, it cannot rely upon the
     processor family to determine if the workaround for AMD Family 10h Erratum
     383 should be enabled.  To enable virtual machine migration among a
     heterogeneous collection of physical machines, the hypervisor may have
     been configured to report an older processor family with a reduced feature
     set.  Effectively, the reported processor family and its features are like
     a "least common denominator" for the collection of machines.
   
     Therefore, when the kernel is running in a virtual machine, instead of
     relying upon the processor family, we now test for features that prove
     that the underlying processor is not affected by the erratum.  (The
     features that we test for are unlikely to ever be emulated in software
     on an affected physical processor.)
   
   PR:		186061
 
 Modified:
   stable/8/sys/amd64/amd64/pmap.c
   stable/8/sys/i386/i386/pmap.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/   (props changed)
   stable/8/sys/i386/   (props changed)
 
 Modified: stable/8/sys/amd64/amd64/pmap.c
 ==============================================================================
 --- stable/8/sys/amd64/amd64/pmap.c	Wed May  7 16:16:49 2014	(r265555)
 +++ stable/8/sys/amd64/amd64/pmap.c	Wed May  7 16:28:36 2014	(r265556)
 @@ -706,12 +706,18 @@ pmap_init(void)
  	}
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 
 Modified: stable/8/sys/i386/i386/pmap.c
 ==============================================================================
 --- stable/8/sys/i386/i386/pmap.c	Wed May  7 16:16:49 2014	(r265555)
 +++ stable/8/sys/i386/i386/pmap.c	Wed May  7 16:28:36 2014	(r265556)
 @@ -757,12 +757,18 @@ pmap_init(void)
  	pv_entry_high_water = 9 * (pv_entry_max / 10);
  
  	/*
 -	 * If the kernel is running in a virtual machine on an AMD Family 10h
 -	 * processor, then it must assume that MCA is enabled by the virtual
 -	 * machine monitor.
 -	 */
 -	if (vm_guest == VM_GUEST_VM && cpu_vendor_id == CPU_VENDOR_AMD &&
 -	    CPUID_TO_FAMILY(cpu_id) == 0x10)
 +	 * If the kernel is running on a virtual machine, then it must assume
 +	 * that MCA is enabled by the hypervisor.  Moreover, the kernel must
 +	 * be prepared for the hypervisor changing the vendor and family that
 +	 * are reported by CPUID.  Consequently, the workaround for AMD Family
 +	 * 10h Erratum 383 is enabled if the processor's feature set does not
 +	 * include at least one feature that is only supported by older Intel
 +	 * or newer AMD processors.
 +	 */
 +	if (vm_guest == VM_GUEST_VM && (cpu_feature & CPUID_SS) == 0 &&
 +	    (cpu_feature2 & (CPUID2_SSSE3 | CPUID2_SSE41 | CPUID2_AESNI |
 +	    CPUID2_AVX | CPUID2_XSAVE)) == 0 && (amd_feature2 & (AMDID2_XOP |
 +	    AMDID2_FMA4)) == 0)
  		workaround_erratum383 = 1;
  
  	/*
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
