From g.olgeni@colby.it  Sun Aug  5 09:34:30 2012
Return-Path: <g.olgeni@colby.it>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 29D8F106566B;
	Sun,  5 Aug 2012 09:34:30 +0000 (UTC)
	(envelope-from g.olgeni@colby.it)
Received: from mail.colby.tv (93-62-141-58.ip22.fastwebnet.it [93.62.141.58])
	by mx1.freebsd.org (Postfix) with ESMTP id A22328FC0A;
	Sun,  5 Aug 2012 09:34:28 +0000 (UTC)
Received: from server.colby.local (localhost [127.0.0.1])
	by server.colby.local (8.14.5/8.14.5) with ESMTP id q759VBNl028873;
	Sun, 5 Aug 2012 11:31:11 +0200 (CEST)
	(envelope-from g.olgeni@colby.it)
Received: from exchange.colby.local ([192.168.1.11] helo=exchange.colby.local)
	with IPv4:25 by server.colby.local; 5 Aug 2012 11:31:11 +0200
Received: from backoffice.colby.local ([192.168.1.56]) by exchange.colby.local over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675);
	 Sun, 5 Aug 2012 11:31:11 +0200
Received: from backoffice.colby.local (localhost [127.0.0.1])
	by backoffice.colby.local (8.14.5/8.14.5) with ESMTP id q759VBK7035003;
	Sun, 5 Aug 2012 11:31:11 +0200 (CEST)
	(envelope-from olgeni@backoffice.colby.local)
Received: (from olgeni@localhost)
	by backoffice.colby.local (8.14.5/8.14.5/Submit) id q759VBVD035002;
	Sun, 5 Aug 2012 11:31:11 +0200 (CEST)
	(envelope-from olgeni)
Message-Id: <201208050931.q759VBVD035002@backoffice.colby.local>
Date: Sun, 5 Aug 2012 11:31:11 +0200 (CEST)
From: Jimmy Olgeni <olgeni@freebsd.org>
Reply-To: Jimmy Olgeni <olgeni@freebsd.org>
To: FreeBSD-gnats-submit@freebsd.org
Cc: jkim@freebsd.org
Subject: 8-STABLE amd64 past r233799 is unable to boot in certain KVM environments
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         170388
>Category:       amd64
>Synopsis:       8-STABLE amd64 past r233799 is unable to boot in certain KVM environments [regression]
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kib
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Aug 05 09:40:02 UTC 2012
>Closed-Date:    Sat Sep 08 16:52:58 UTC 2012
>Last-Modified:  Sat Sep 08 16:52:58 UTC 2012
>Originator:     Jimmy Olgeni
>Release:        FreeBSD 8.3-STABLE amd64
>Organization:
>Environment:
>Description:

FreeBSD 8-STABLE is unable to boot in some KVM environments after r233799.

------------------------------------------------------------------------
r233799 | jkim | 2012-04-02 20:27:06 +0200 (Mon, 02 Apr 2012) | 4 lines

MFC:    r233702

Work around Erratum 721 for AMD Family 10h and 12h processors.

------------------------------------------------------------------------

The following panic message is shown immediately after the kernel starts:

===
kernel trap 9 with interrupts disabled

Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer    = 0x20:0xffffffff808fd8dc
stack pointer          = 0x28:0xffffffff80cd5440
frame pointer          = 0x28:0xffffffff80cd5470
code segment           = base 0x0,  limit 0xfffff, type 0x1b
                       = DPL 0, pres 1,  long 1, def32 0, gran 1
processor eflags       = resume,  IOPL = 0
current process        = 0 ()
trap number            = 9
panic: general protection fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff8064e18e at ??+0
#1 0xffffffff8061b247 at ??+0
#2 0xffffffff80912ca0 at ??+0
#3 0xffffffff80913235 at ??+0
#4 0xffffffff808faad4 at ??+0
#5 0xffffffff80904cc2 at ??+0
#6 0xffffffff801a0244 at ??+0
===

When booting from the latest 8-STABLE branch the booting process
just halts after the first panic line.

The same problem can be observed in 9-STABLE, but I did not actually
bisect all the way to the relevant commit yet. However, r233702
seems a reasonable guess.

Once the system is booted by reverting r233799, the following
processor information can be read:

    CPU: Six-Core AMD Opteron(tm) Processor 2427 (2211.49-MHz K8-class CPU)
      Origin = "AuthenticAMD"  Id = 0x100f80  Family = 10  Model = 8  Stepping = 0
      Features=0x783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2>
      Features2=0x80802001<SSE3,CX16,POPCNT,HV>
      AMD Features=0xe6500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,LM,3DNow!+,3DNow!>
      AMD Features2=0x1f7<LAHF,CMP,SVM,CR8,ABM,SSE4A,MAS,Prefetch>
      TSC: P-state invariant

Also, using x86info:

    x86info v1.30.  Dave Jones 2001-2011
    Feedback to <davej@redhat.com>.

    Extended Family: 1 Extended Model: 0 Family: 15 Model: 8 Stepping: 0
    CPU Model (x86info's best guess): Phenom/Athlon/Sempron/Turion (II)/Opteron (HY-D0)
    Processor name string (BIOS programmed): Six-Core AMD Opteron(tm) Processor 2427

    Monitor/Mwait: min/max line size 0/0, ecx bit 0 support, enumeration extension
    SVM: revision 1, 16 ASIDs, np, NRIPSave
    Address Size: 48 bits virtual, 40 bits physical
    running at an estimated 2.35GHz

Unfortunately I have no detailed information about the QEMU
configuration, but I set up a dedicated VM that I could use to test
patches.

>How-To-Repeat:
>Fix:
>Release-Note:
>Audit-Trail:

From: Konstantin Belousov <kostikbel@gmail.com>
To: Jimmy Olgeni <olgeni@freebsd.org>
Cc: FreeBSD-gnats-submit@freebsd.org, jkim@freebsd.org
Subject: Re: amd64/170388: 8-STABLE amd64 past r233799 is unable to boot in certain KVM environments
Date: Sun, 5 Aug 2012 16:55:18 +0300

 --USJBhEEH8TT+Is94
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable
 
 Try this. Comment should give enough explanation what happens there, my
 guess anyway.
 
 diff --git a/sys/amd64/amd64/initcpu.c b/sys/amd64/amd64/initcpu.c
 index 3890551..dbeaec6 100644
 --- a/sys/amd64/amd64/initcpu.c
 +++ b/sys/amd64/amd64/initcpu.c
 @@ -91,11 +91,17 @@ init_amd(void)
  	 *
  	 * http://support.amd.com/us/Processor_TechDocs/41322_10h_Rev_Gd.pdf
  	 * http://support.amd.com/us/Processor_TechDocs/44739_12h_Rev_Gd.pdf
 +	 *
 +	 * Hypervisors do not provide access to the errata MSR,
 +	 * causing #GP exception on attempt to apply the errata.  The
 +	 * MSR write shall be done on host and persist globally
 +	 * anyway, so do not try to do it when under virtualization.
  	 */
  	switch (CPUID_TO_FAMILY(cpu_id)) {
  	case 0x10:
  	case 0x12:
 -		wrmsr(0xc0011029, rdmsr(0xc0011029) | 1);
 +		if ((cpu_feature2 & CPUID2_HV) =3D=3D 0)
 +			wrmsr(0xc0011029, rdmsr(0xc0011029) | 1);
  		break;
  	}
  }
 
 
 --USJBhEEH8TT+Is94
 Content-Type: application/pgp-signature
 Content-Disposition: inline
 
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.12 (FreeBSD)
 
 iEYEARECAAYFAlAee0UACgkQC3+MBN1Mb4gWWACg6fxUCW+uah3gPZZxiqTEtH8j
 G+oAoMXjm4aBYfLr8af7cCc3PwWL/kXX
 =yeJJ
 -----END PGP SIGNATURE-----
 
 --USJBhEEH8TT+Is94--

From: Jimmy Olgeni <olgeni@FreeBSD.org>
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: jkim@FreeBSD.org, bug-followup@FreeBSD.org
Subject: Re: amd64/170388: 8-STABLE amd64 past r233799 is unable to boot in
 certain KVM environments
Date: Sun, 5 Aug 2012 20:05:34 +0200 (CEST)

 On Sun, 5 Aug 2012, Konstantin Belousov wrote:
 
 > Try this. Comment should give enough explanation what happens there, my
 > guess anyway.
 
 Works great: I just booted into 8-STABLE.
 
 I'll try with 9-STABLE too - rebuilding right now.
 
 Thanks!

From: olgeni@FreeBSD.org
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: jkim@FreeBSD.org, bug-followup@FreeBSD.org
Subject: Re: amd64/170388: 8-STABLE amd64 past r233799 is unable to boot in
 certain KVM environments
Date: Mon, 6 Aug 2012 01:27:48 +0200 (CEST)

 On Sun, 5 Aug 2012, Konstantin Belousov wrote:
 
 > Try this. Comment should give enough explanation what happens there, my
 > guess anyway.
 
 Works great on 9-STABLE too (tested on r239079).
 
 Looks like a *great* candidate for 9.1.
 
 -- 
 jimmy
Responsible-Changed-From-To: freebsd-amd64->kib 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Aug 6 09:20:26 UTC 2012 
Responsible-Changed-Why:  
kib has a patch and submitter notes it fixes the problem. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=170388 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/170388: commit references a PR
Date: Tue,  7 Aug 2012 08:36:20 +0000 (UTC)

 Author: kib
 Date: Tue Aug  7 08:36:10 2012
 New Revision: 239125
 URL: http://svn.freebsd.org/changeset/base/239125
 
 Log:
   Do not apply errata 721 workaround when under hypervisor, since
   typical hypervisor does not implement access to the required MSR,
   causing #GP on boot.
   
   Reported and tested by:	olgeni
   PR:	amd64/170388
   MFC after:	3 days
 
 Modified:
   head/sys/amd64/amd64/initcpu.c
 
 Modified: head/sys/amd64/amd64/initcpu.c
 ==============================================================================
 --- head/sys/amd64/amd64/initcpu.c	Tue Aug  7 07:52:25 2012	(r239124)
 +++ head/sys/amd64/amd64/initcpu.c	Tue Aug  7 08:36:10 2012	(r239125)
 @@ -91,11 +91,17 @@ init_amd(void)
  	 *
  	 * http://support.amd.com/us/Processor_TechDocs/41322_10h_Rev_Gd.pdf
  	 * http://support.amd.com/us/Processor_TechDocs/44739_12h_Rev_Gd.pdf
 +	 *
 +	 * Hypervisors do not provide access to the errata MSR,
 +	 * causing #GP exception on attempt to apply the errata.  The
 +	 * MSR write shall be done on host and persist globally
 +	 * anyway, so do not try to do it when under virtualization.
  	 */
  	switch (CPUID_TO_FAMILY(cpu_id)) {
  	case 0x10:
  	case 0x12:
 -		wrmsr(0xc0011029, rdmsr(0xc0011029) | 1);
 +		if ((cpu_feature2 & CPUID2_HV) == 0)
 +			wrmsr(0xc0011029, rdmsr(0xc0011029) | 1);
  		break;
  	}
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: olgeni@FreeBSD.org
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: amd64/170388: 8-STABLE amd64 past r233799 is unable to boot in
 certain KVM environments
Date: Sun, 19 Aug 2012 14:04:08 +0200 (CEST)

 Hello,
 
 Just a quick reminder about the MFC of r239125 :)
 
 Thanks!
 
 -- 
 jimmy

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: amd64/170388: commit references a PR
Date: Sat,  8 Sep 2012 16:48:06 +0000 (UTC)

 Author: kib
 Date: Sat Sep  8 16:47:52 2012
 New Revision: 240240
 URL: http://svn.freebsd.org/changeset/base/240240
 
 Log:
   MFC r239125:
   Do not apply errata 721 workaround when under hypervisor, since
   typical hypervisor does not implement access to the required MSR,
   causing #GP on boot.
   
   PR:	amd64/170388
   Approved by:	re (kensmith)
 
 Modified:
   releng/9.1/sys/amd64/amd64/initcpu.c
 Directory Properties:
   releng/9.1/sys/   (props changed)
 
 Modified: releng/9.1/sys/amd64/amd64/initcpu.c
 ==============================================================================
 --- releng/9.1/sys/amd64/amd64/initcpu.c	Sat Sep  8 16:45:48 2012	(r240239)
 +++ releng/9.1/sys/amd64/amd64/initcpu.c	Sat Sep  8 16:47:52 2012	(r240240)
 @@ -91,11 +91,17 @@ init_amd(void)
  	 *
  	 * http://support.amd.com/us/Processor_TechDocs/41322_10h_Rev_Gd.pdf
  	 * http://support.amd.com/us/Processor_TechDocs/44739_12h_Rev_Gd.pdf
 +	 *
 +	 * Hypervisors do not provide access to the errata MSR,
 +	 * causing #GP exception on attempt to apply the errata.  The
 +	 * MSR write shall be done on host and persist globally
 +	 * anyway, so do not try to do it when under virtualization.
  	 */
  	switch (CPUID_TO_FAMILY(cpu_id)) {
  	case 0x10:
  	case 0x12:
 -		wrmsr(0xc0011029, rdmsr(0xc0011029) | 1);
 +		if ((cpu_feature2 & CPUID2_HV) == 0)
 +			wrmsr(0xc0011029, rdmsr(0xc0011029) | 1);
  		break;
  	}
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: kib 
State-Changed-When: Sat Sep 8 16:52:27 UTC 2012 
State-Changed-Why:  
Merged. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=170388 
>Unformatted:
