From nobody@FreeBSD.org  Thu Nov 11 22:46:36 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 263501065675
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Nov 2010 22:46:36 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 1526C8FC1B
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Nov 2010 22:46:36 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id oABMkZer028045
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 11 Nov 2010 22:46:35 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id oABMkZ19028044;
	Thu, 11 Nov 2010 22:46:35 GMT
	(envelope-from nobody)
Message-Id: <201011112246.oABMkZ19028044@www.freebsd.org>
Date: Thu, 11 Nov 2010 22:46:35 GMT
From: Rob Farmer <rfarmer@predatorlabs.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: nfe(4) not working on Asus P5N32-SLI Premium motherboard (nforce 590)
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         152150
>Category:       kern
>Synopsis:       nfe(4) not working on Asus P5N32-SLI Premium motherboard (nforce 590)
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    yongari
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Nov 11 22:50:09 UTC 2010
>Closed-Date:    Tue Nov 23 19:39:10 UTC 2010
>Last-Modified:  Tue Nov 23 19:39:10 UTC 2010
>Originator:     Rob Farmer
>Release:        9.0-CURRENT
>Organization:
>Environment:
FreeBSD amethyst.predatorlabs.net 9.0-CURRENT FreeBSD 9.0-CURRENT #1 r215105: Thu Nov 11 11:31:16 PST 2010     rfarmer@amethyst.predatorlabs.net:/usr/obj/usr/src/sys/AMETHYST  amd64
>Description:
On Asus P5N32-SLI Premium motherboard (nforce 590), nfe driver repeats:

watchdog timeout (missed Tx interrupts)

and will not get an IP from DHCP. Setting a static IP doesn't work either (128.95.133.100 is the gateway):
ping 128.95.133.100
PING 128.95.133.100 (128.95.133.100): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- 128.95.133.100 ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss

Also, it also generates so much bogus traffic that all other systems connected to the same switch/hub lose their network connection while the machine is running (I don't know how to troubleshoot this part better, sorry).

nve driver works fine.

>How-To-Repeat:
Use nfe(4) driver on said board
>Fix:


>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback 
State-Changed-By: yongari 
State-Changed-When: Thu Nov 11 23:59:50 UTC 2010 
State-Changed-Why:  
It seems you have three issues here. 
o watchdog timeout issue 
o can't send/receive frames with either DHCP and static IP assignment 
o generate bogus traffic storm which in turn make systems connected to 
switch lost connection  

For watchdog timeout issue, would you try the patch at the 
following URL? 
http://people.freebsd.org/~yongari/nfe/nfe.watchdog.diff 

The second issue could be related with tracking of link state 
change. The patch above will show "LINK UP" message on your console 
if nfe(4) think it established a link. Otherwise it would print 
"LINK DOWN". In order to send/receive frames, you always have to 
see "LINK UP" message first. Please check whether you can see the 
message with DHCP or static IP configuration. 

I have no idea for the last issue. Can you see what kind of traffic 
is generated on link partner side? I think you can connect two 
systems with straight cable and see what kind of traffic does the 
nfe(4) generate on other system with tcpdump. If it's 802.3x flow 
control traffic I have idea how to keep it from happening. 


Responsible-Changed-From-To: freebsd-bugs->yongari 
Responsible-Changed-By: yongari 
Responsible-Changed-When: Thu Nov 11 23:59:50 UTC 2010 
Responsible-Changed-Why:  
Grab. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152150 

From: Rob Farmer <rfarmer@predatorlabs.net>
To: bug-followup@FreeBSD.org, rfarmer@predatorlabs.net
Cc:  
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Thu, 11 Nov 2010 21:20:57 -0800

 r215135 + patch panics on attempt to bring up the interface
 
 http://www.predatorlabs.net/dl/textdump.tar.1
 
 -- 
 Rob Farmer

From: Pyun YongHyeon <pyunyh@gmail.com>
To: Rob Farmer <rfarmer@predatorlabs.net>
Cc: yongari@freebsd.org, bug-followup@FreeBSD.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium motherboard (nforce 590)
Date: Fri, 12 Nov 2010 11:40:01 -0800

 On Fri, Nov 12, 2010 at 05:30:13AM +0000, Rob Farmer wrote:
 > The following reply was made to PR kern/152150; it has been noted by GNATS.
 > 
 > From: Rob Farmer <rfarmer@predatorlabs.net>
 > To: bug-followup@FreeBSD.org, rfarmer@predatorlabs.net
 > Cc:  
 > Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 >  motherboard (nforce 590)
 > Date: Thu, 11 Nov 2010 21:20:57 -0800
 > 
 >  r215135 + patch panics on attempt to bring up the interface
 >  
 >  http://www.predatorlabs.net/dl/textdump.tar.1
 >  
 
 Thanks for reporting. Actually it was regression introduced in
 r215132. I committed fix(r215194) and regenerated a new patch to
 try against latest HEAD.
 
 http://people.freebsd.org/~yongari/nfe/nfe.watchdog.diff
 
 
 >  -- 
 >  Rob Farmer

From: Rob Farmer <rfarmer@predatorlabs.net>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Fri, 12 Nov 2010 22:08:07 -0800

 On Fri, Nov 12, 2010 at 11:40, Pyun YongHyeon <pyunyh@gmail.com> wrote:
 > Thanks for reporting. Actually it was regression introduced in
 > r215132. I committed fix(r215194) and regenerated a new patch to
 > try against latest HEAD.
 >
 > http://people.freebsd.org/~yongari/nfe/nfe.watchdog.diff
 
 I connected the two systems directly with one cable, and set static
 IPs of 192.168.1.1 for the good system and 192.168.1.2 for the problem
 one. I get both "LINK UP" and "nfe0: link state changed to UP" on the
 console. I ran tcpdump on the good system and tried pinging it from
 the bad one, as well as login with ssh. It looks to me like the bad
 system can send traffic, but can't respond. I didn't see anything that
 represents the flood of activity, though.
 
 tcpdump with nfe driver:
 http://www.predatorlabs.net/dl/nfe/nfe.txt
 
 with nve driver (working):
 http://www.predatorlabs.net/dl/nfe/nve.txt
 
 If I put the switch in the middle (but don't connect upstream), then
 nothing happens for about 90 seconds. From there, tcpdump output is
 the same as with a direct connection:
 http://www.predatorlabs.net/dl/nfe/nfe_switch.txt
 
 And here is from the bad system while trying to ping:
 http://www.predatorlabs.net/dl/nfe/bad.txt
 
 -- 
 Rob Farmer

From: Rob Farmer <rfarmer@predatorlabs.net>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Fri, 12 Nov 2010 22:37:59 -0800

 I think I have solved this - I looked at the Linux driver
 (forcedeth.c) and it says that MSI is broken for this model. So I set
 hw.pci.enable_msi="0" and hw.pci.enable_msix="0" and now it seems to
 be working.
 
 But I don't know what MSI is - is this a bad idea?
 
 -- 
 Rob Farmer

From: Pyun YongHyeon <pyunyh@gmail.com>
To: Rob Farmer <rfarmer@predatorlabs.net>
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Fri, 12 Nov 2010 23:15:49 -0800

 On Fri, Nov 12, 2010 at 10:37 PM, Rob Farmer <rfarmer@predatorlabs.net> wrote:
 > I think I have solved this - I looked at the Linux driver
 > (forcedeth.c) and it says that MSI is broken for this model. So I set
 > hw.pci.enable_msi="0" and hw.pci.enable_msix="0" and now it seems to
 > be working.
 >
 
 Interesting. Could you show me the output of "sysctl smbios"?
 Also show me the output of "pciconf -lcbv".
 
 > But I don't know what MSI is - is this a bad idea?
 >
 
 Without MSI/MSI-X your NIC shall share interrupt with other
 devices such that sometimes it could take inefficient interrupt
 handling path but most cases it wouldn't affect normal
 operation.
 
 Correct workaround would be checking motherboard specific
 information from SMBIOS. Show me output I mentioned above.
 
 > --
 > Rob Farmer
 >

From: Rob Farmer <rfarmer@predatorlabs.net>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Sat, 13 Nov 2010 12:34:15 -0800

 On Fri, Nov 12, 2010 at 23:15, Pyun YongHyeon <pyunyh@gmail.com> wrote:
 > Interesting. Could you show me the output of "sysctl smbios"?
 
 Where does that come from? I get sysctl: unknown oid 'smbios'
 
 I built a new kernel with "device smbios" and only got:
 
 dev.smbios.0.%desc: System Management BIOS
 dev.smbios.0.%driver: smbios
 dev.smbios.0.%parent: nexus0
 
 > Also show me the output of "pciconf -lcbv".
 
 hostb0@pci0:0:0:0:	class=0x060000 card=0x00000000 chip=0x007110de
 rev=0xc1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition CPU to PCI Bridge'
     class      = bridge
     subclass   = HOST-PCI
     cap 08[40] = HT host
 none0@pci0:0:0:1:	class=0x050000 card=0x00000000 chip=0x007f10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none1@pci0:0:0:2:	class=0x050000 card=0x00000000 chip=0x007510de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none2@pci0:0:0:3:	class=0x050000 card=0x00000000 chip=0x006f10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none3@pci0:0:0:4:	class=0x050000 card=0x00000000 chip=0x00b410de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none4@pci0:0:1:0:	class=0x050000 card=0x00000000 chip=0x007610de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none5@pci0:0:1:1:	class=0x050000 card=0x00000000 chip=0x007810de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none6@pci0:0:1:2:	class=0x050000 card=0x00000000 chip=0x007910de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none7@pci0:0:1:3:	class=0x050000 card=0x00000000 chip=0x007a10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none8@pci0:0:1:4:	class=0x050000 card=0x00000000 chip=0x007b10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none9@pci0:0:1:5:	class=0x050000 card=0x00000000 chip=0x007c10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none10@pci0:0:1:6:	class=0x050000 card=0x00000000 chip=0x007d10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 pcib1@pci0:0:2:0:	class=0x060400 card=0x00000000 chip=0x007e10de
 rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition PCIe Root Hub'
     class      = bridge
     subclass   = PCI-PCI
     cap 01[40] = powerspec 2  supports D0 D3  current D0
     cap 05[48] = MSI supports 2 messages, 64 bit
     cap 10[80] = PCI-Express 1 root port max data 128(128) link x16(x16)
 ecap 0002[100] = VC 1 max VC1 lowpri VC0-VC1
 pcib2@pci0:0:5:0:	class=0x060400 card=0x00000000 chip=0x007e10de
 rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition PCIe Root Hub'
     class      = bridge
     subclass   = PCI-PCI
     cap 01[40] = powerspec 2  supports D0 D3  current D0
     cap 05[48] = MSI supports 2 messages, 64 bit
     cap 10[80] = PCI-Express 1 root port max data 128(128) link x1(x1)
 ecap 0002[100] = VC 1 max VC1 lowpri VC0-VC1
 none11@pci0:0:9:0:	class=0x050000 card=0xcb841043 chip=0x036910de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Memory Controller'
     class      = memory
     subclass   = RAM
     cap 08[44] = HT slave
     cap 00[e0] = unknown
 isab0@pci0:0:10:0:	class=0x060100 card=0xcb841043 chip=0x036010de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 LPC Bridge'
     class      = bridge
     subclass   = PCI-ISA
 none12@pci0:0:10:1:	class=0x0c0500 card=0xcb841043 chip=0x036810de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'SMBus controller ((0xCB84 integrated chip nForce Pro 3400))'
     class      = serial bus
     subclass   = SMBus
     bar   [10] = type I/O Port, range 32, base 0xfc00, size 64, enabled
     bar   [20] = type I/O Port, range 32, base 0x1c00, size 64, enabled
     bar   [24] = type I/O Port, range 32, base 0x1c80, size 64, enabled
     cap 01[44] = powerspec 2  supports D0 D3  current D0
 none13@pci0:0:10:2:	class=0x050000 card=0xcb841043 chip=0x036a10de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Memory Controller'
     class      = memory
     subclass   = RAM
 ohci0@pci0:0:11:0:	class=0x0c0310 card=0xcb841043 chip=0x036c10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'Standard OpenHCD USB Hostcontroller (81FB1043)'
     class      = serial bus
     subclass   = USB
     bar   [10] = type Memory, range 32, base 0xfe02f000, size 4096, enabled
     cap 01[44] = powerspec 2  supports D0 D1 D2 D3  current D0
 ehci0@pci0:0:11:1:	class=0x0c0320 card=0xcb841043 chip=0x036d10de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'Standard PCI-to-USB Enhanced Hostcontroller (81FB1043)'
     class      = serial bus
     subclass   = USB
     bar   [10] = type Memory, range 32, base 0xfe02e000, size 256, enabled
     cap 0a[44] = EHCI Debug Port at offset 0x98 in map 0x14
     cap 01[80] = powerspec 2  supports D0 D1 D2 D3  current D0
 atapci1@pci0:0:13:0:	class=0x01018a card=0xcb841043 chip=0x036e10de
 rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PATA Controller (MCP55)'
     class      = mass storage
     subclass   = ATA
     bar   [20] = type I/O Port, range 32, base 0xf000, size 16, enabled
     cap 01[44] = powerspec 2  supports D0 D3  current D0
 atapci2@pci0:0:14:0:	class=0x010185 card=0xcb841043 chip=0x037f10de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 SATA/RAID Controller (MCP55S)'
     class      = mass storage
     subclass   = ATA
     bar   [10] = type I/O Port, range 32, base 0x9f0, size  8, enabled
     bar   [14] = type I/O Port, range 32, base 0xbf0, size  4, enabled
     bar   [18] = type I/O Port, range 32, base 0x970, size  8, enabled
     bar   [1c] = type I/O Port, range 32, base 0xb70, size  4, enabled
     bar   [20] = type I/O Port, range 32, base 0xdc00, size 16, enabled
     bar   [24] = type Memory, range 32, base 0xfe02d000, size 4096, enabled
     cap 01[44] = powerspec 2  supports D0 D3  current D0
     cap 05[b0] = MSI supports 4 messages, 64 bit
     cap 08[cc] = HT MSI fixed address window disabled at 0xfee00000
 atapci3@pci0:0:14:1:	class=0x010185 card=0xcb841043 chip=0x037f10de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 SATA/RAID Controller (MCP55S)'
     class      = mass storage
     subclass   = ATA
     bar   [10] = type I/O Port, range 32, base 0x9e0, size  8, enabled
     bar   [14] = type I/O Port, range 32, base 0xbe0, size  4, enabled
     bar   [18] = type I/O Port, range 32, base 0x960, size  8, enabled
     bar   [1c] = type I/O Port, range 32, base 0xb60, size  4, enabled
     bar   [20] = type I/O Port, range 32, base 0xc800, size 16, enabled
     bar   [24] = type Memory, range 32, base 0xfe02c000, size 4096, enabled
     cap 01[44] = powerspec 2  supports D0 D3  current D0
     cap 05[b0] = MSI supports 4 messages, 64 bit
     cap 08[cc] = HT MSI fixed address window disabled at 0xfee00000
 atapci4@pci0:0:14:2:	class=0x010185 card=0xcb841043 chip=0x037f10de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 SATA/RAID Controller (MCP55S)'
     class      = mass storage
     subclass   = ATA
     bar   [10] = type I/O Port, range 32, base 0xc400, size  8, enabled
     bar   [14] = type I/O Port, range 32, base 0xc000, size  4, enabled
     bar   [18] = type I/O Port, range 32, base 0xbc00, size  8, enabled
     bar   [1c] = type I/O Port, range 32, base 0xb800, size  4, enabled
     bar   [20] = type I/O Port, range 32, base 0xb400, size 16, enabled
     bar   [24] = type Memory, range 32, base 0xfe02b000, size 4096, enabled
     cap 01[44] = powerspec 2  supports D0 D3  current D0
     cap 05[b0] = MSI supports 4 messages, 64 bit
     cap 08[cc] = HT MSI fixed address window disabled at 0xfee00000
 pcib3@pci0:0:15:0:	class=0x060401 card=0xcb8410de chip=0x037010de
 rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCI bridge'
     class      = bridge
     subclass   = PCI-PCI
     cap 0d[b8] = PCI Bridge card=0xcb8410de
     cap 08[8c] = HT MSI address window disabled at 0xfee00000
 hdac0@pci0:0:15:1:	class=0x040300 card=0x81f61043 chip=0x037110de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'High Definition Audio Controller (MCP55)'
     class      = multimedia
     subclass   = HDA
     bar   [10] = type Memory, range 32, base 0xfe020000, size 16384, enabled
     cap 01[44] = powerspec 2  supports D0 D3  current D0
     cap 05[50] = MSI supports 1 message, 64 bit, vector masks
     cap 08[6c] = HT MSI fixed address window disabled at 0xfee00000
 nfe0@pci0:0:17:0:	class=0x068000 card=0xcb841043 chip=0x037310de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Ethernet'
     class      = bridge
     bar   [10] = type Memory, range 32, base 0xfe02a000, size 4096, enabled
     bar   [14] = type I/O Port, range 32, base 0xb000, size  8, enabled
     bar   [18] = type Memory, range 32, base 0xfe029000, size 256, enabled
     bar   [1c] = type Memory, range 32, base 0xfe028000, size 16, enabled
     cap 01[44] = powerspec 2  supports D0 D1 D2 D3  current D0
     cap 11[70] = MSI-X supports 8 messages in maps 0x18 and 0x1c
     cap 05[50] = MSI supports 8 messages, 64 bit, vector masks
     cap 08[6c] = HT MSI fixed address window disabled at 0xfee00000
 nfe1@pci0:0:18:0:	class=0x068000 card=0xcb841043 chip=0x037310de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Ethernet'
     class      = bridge
     bar   [10] = type Memory, range 32, base 0xfe027000, size 4096, enabled
     bar   [14] = type I/O Port, range 32, base 0xac00, size  8, enabled
     bar   [18] = type Memory, range 32, base 0xfe026000, size 256, enabled
     bar   [1c] = type Memory, range 32, base 0xfe025000, size 16, enabled
     cap 01[44] = powerspec 2  supports D0 D1 D2 D3  current D0
     cap 11[70] = MSI-X supports 8 messages in maps 0x18 and 0x1c
     cap 05[50] = MSI supports 8 messages, 64 bit, vector masks
     cap 08[6c] = HT MSI fixed address window disabled at 0xfee00000
 pcib4@pci0:0:19:0:	class=0x060400 card=0x000010de chip=0x037610de
 rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCIe bridge'
     class      = bridge
     subclass   = PCI-PCI
     cap 0d[40] = PCI Bridge card=0x000010de
     cap 01[48] = powerspec 2  supports D0 D3  current D0
     cap 05[50] = MSI supports 2 messages, 64 bit
     cap 08[60] = HT MSI address window disabled at 0xfee00000
     cap 10[80] = PCI-Express 1 root port max data 256(256) link x8(x8)
 ecap 0002[100] = VC 1 max VC0
 pcib5@pci0:0:23:0:	class=0x060400 card=0x000010de chip=0x037510de
 rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCIe bridge'
     class      = bridge
     subclass   = PCI-PCI
     cap 0d[40] = PCI Bridge card=0x000010de
     cap 01[48] = powerspec 2  supports D0 D3  current D0
     cap 05[50] = MSI supports 2 messages, 64 bit
     cap 08[60] = HT MSI address window disabled at 0xfee00000
     cap 10[80] = PCI-Express 1 root port max data 256(256) link x8(x1)
 ecap 0002[100] = VC 1 max VC0
 pcib6@pci0:0:24:0:	class=0x060400 card=0x000010de chip=0x037710de
 rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCIe bridge'
     class      = bridge
     subclass   = PCI-PCI
     cap 0d[40] = PCI Bridge card=0x000010de
     cap 01[48] = powerspec 2  supports D0 D3  current D0
     cap 05[50] = MSI supports 2 messages, 64 bit
     cap 08[60] = HT MSI address window disabled at 0xfee00000
     cap 10[80] = PCI-Express 1 root port max data 128(256) link x16(x16)
 ecap 0002[100] = VC 1 max VC0
 vgapci0@pci0:1:0:0:	class=0x030000 card=0x042010de chip=0x019310de
 rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'NVIDIA GeForce 8800 GTS (G80)'
     class      = display
     subclass   = VGA
     bar   [10] = type Memory, range 32, base 0xfa000000, size 16777216, enabled
     bar   [14] = type Prefetchable Memory, range 64, base 0xd0000000,
 size 268435456, enabled
     bar   [1c] = type Memory, range 64, base 0xf8000000, size 33554432, enabled
     bar   [24] = type I/O Port, range 32, base 0x8c00, size 128, enabled
     cap 01[60] = powerspec 2  supports D0 D3  current D0
     cap 05[68] = MSI supports 1 message, 64 bit
     cap 10[78] = PCI-Express 1 endpoint max data 128(128) link x16(x16)
 ecap 0002[100] = VC 1 max VC0
 ecap 0004[128] = unknown 1
 atapci0@pci0:2:0:0:	class=0x018000 card=0x819f1043 chip=0x31321095
 rev=0x01 hdr=0x00
     vendor     = 'Silicon Image Inc (Was: CMD Technology Inc)'
     device     = 'PCI Express (1x) to 2 Port SATA300 (SiI 3132)'
     class      = mass storage
     bar   [10] = type Memory, range 64, base 0xfdfff000, size 128, enabled
     bar   [18] = type Memory, range 64, base 0xfdff8000, size 16384, enabled
     bar   [20] = type I/O Port, range 32, base 0x9c00, size 128, enabled
     cap 01[54] = powerspec 2  supports D0 D1 D2 D3  current D0
     cap 05[5c] = MSI supports 1 message, 64 bit
     cap 10[70] = PCI-Express 1 legacy endpoint max data 128(1024) link x1(x1)
 ecap 0001[100] = AER 1 0 fatal 1 non-fatal 0 corrected
 none14@pci0:3:7:0:	class=0x040000 card=0x74440070 chip=0x5b7a14f1
 rev=0x00 hdr=0x00
     vendor     = 'Conexant Systems, Inc.'
     device     = 'Single-Chip MPEG-2 Encoder with Integrated Analog
 Video/Broadcast Audio Decoder (Belived to be a CX23416)'
     class      = multimedia
     subclass   = video
     bar   [10] = type Memory, range 32, base 0xf4000000, size 67108864, enabled
     cap 03[44] = VPD
     cap 01[4c] = powerspec 2  supports D0 D3  current D0
 
 -- 
 Rob Farmer

From: Pyun YongHyeon <pyunyh@gmail.com>
To: Rob Farmer <rfarmer@predatorlabs.net>
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Sat, 13 Nov 2010 17:10:35 -0800

 On Sat, Nov 13, 2010 at 12:34 PM, Rob Farmer <rfarmer@predatorlabs.net> wrote:
 > On Fri, Nov 12, 2010 at 23:15, Pyun YongHyeon <pyunyh@gmail.com> wrote:
 >> Interesting. Could you show me the output of "sysctl smbios"?
 >
 > Where does that come from? I get sysctl: unknown oid 'smbios'
 >
 > I built a new kernel with "device smbios" and only got:
 >
 > dev.smbios.0.%desc: System Management BIOS
 > dev.smbios.0.%driver: smbios
 > dev.smbios.0.%parent: nexus0
 >
 
 Oops, sorry, I should have drunk a cup of coffee here. :-(
 Show me the output of "kenv | grep smbios"
 
 >> Also show me the output of "pciconf -lcbv".
 >
 
 Thanks for the output. Does hdac(4) works with MSI?
 I think you can check the output of "vmstat -i" and see whether
 the interrupt number for hdac(4) is 256 or higher.
 
 Because I don't know whether only MSI-X was broken for nfe(4) or
 all MSI/MSI-X does not work, I think there are 3 choices here.
 o Make MSI/MSI-X capability black-listed in PCI bridge such that all
    drivers on your system wouldn't use MSI/MSI-X anymore.
 o Force nfe(4) to use traditional line interrupt by checking specific
   model number of your system in driver.
 o Update to latest BIOS from Vendor, I don't know whether there
    is newer BIOS releases though. And it wouldn't guarantee Vendor
    released fixed BIOS for the issue.
 
 At this moment, my preference is the second approach. kenv(1)
 output for SMBIOS would be important to identify details of
 motherboard.

From: Rob Farmer <rfarmer@predatorlabs.net>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Sat, 13 Nov 2010 18:32:18 -0800

 On Sat, Nov 13, 2010 at 17:10, Pyun YongHyeon <pyunyh@gmail.com> wrote:
 > Oops, sorry, I should have drunk a cup of coffee here. :-(
 > Show me the output of "kenv | grep smbios"
 
 smbios.bios.reldate=3D"03/12/2008"
 smbios.bios.vendor=3D"Phoenix Technologies, LTD"
 smbios.bios.version=3D"ASUS P5N32-SLI PREMIUM ACPI BIOS Revision 1002"
 smbios.chassis.maker=3D"Chassis Manufacture"
 smbios.chassis.serial=3D"EVAL          "
 smbios.chassis.tag=3D"123456789000"
 smbios.chassis.version=3D"Chassis Version"
 smbios.memory.enabled=3D"4194304"
 smbios.planar.maker=3D"ASUSTeK Computer INC."
 smbios.planar.product=3D"P5N32-SLI PREMIUM"
 smbios.planar.serial=3D"123456789000"
 smbios.planar.version=3D"1.XX    "
 smbios.socket.enabled=3D"1"
 smbios.socket.populated=3D"1"
 smbios.system.maker=3D"System manufacturer"
 smbios.system.product=3D"System Product Name"
 smbios.system.serial=3D"System Serial Number"
 smbios.system.uuid=3D"10079fec-837a-da11-9e12-977b2bdc6f37"
 smbios.system.version=3D"System Version"
 smbios.version=3D"2.4"
 
 > Thanks for the output. Does hdac(4) works with MSI?
 > I think you can check the output of "vmstat -i" and see whether
 > the interrupt number for hdac(4) is 256 or higher.
 
 With MSI:
 interrupt                          total       rate
 irq14: ata0                         2606          7
 irq16: vgapci0+                      629          1
 irq20: nve0 atapci2                94025        273
 irq22: ohci0+                       2883          8
 irq23: hdac0 ehci0                 18352         53
 cpu0:timer                        123602        359
 cpu1:timer                         79227        230
 Total                             321324        934
 
 Without MSI:
 interrupt                          total       rate
 irq14: ata0                         1358          7
 irq16: vgapci0+                      317          1
 irq20: nfe0 atapci2                19971        107
 irq22: ohci0+                       1937         10
 irq23: hdac0 ehci0                 13076         70
 cpu0:timer                         71326        383
 cpu1:timer                         62319        335
 Total                             170304        915
 
 Works fine in both cases.
 
 > Because I don't know whether only MSI-X was broken for nfe(4) or
 > all MSI/MSI-X does not work, I think there are 3 choices here.
 
 Trying just hw.pci.enable_msix=3D"0" doesn't work - I need to disable msi t=
 oo.
 
 > o Make MSI/MSI-X capability black-listed in PCI bridge such that all
 > =A0 drivers on your system wouldn't use MSI/MSI-X anymore.
 > o Force nfe(4) to use traditional line interrupt by checking specific
 > =A0model number of your system in driver.
 > o Update to latest BIOS from Vendor, I don't know whether there
 > =A0 is newer BIOS releases though. And it wouldn't guarantee Vendor
 > =A0 released fixed BIOS for the issue.
 
 There's been no BIOS updates for several years now.
 
 --=20
 Rob Farmer

From: Rob Farmer <rfarmer@predatorlabs.net>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Sat, 13 Nov 2010 18:34:38 -0800

 On Sat, Nov 13, 2010 at 18:32, Rob Farmer <rfarmer@predatorlabs.net> wrote:
 > smbios.chassis.maker=3D"Chassis Manufacture"
 > smbios.chassis.serial=3D"EVAL =A0 =A0 =A0 =A0 =A0"
 > smbios.chassis.tag=3D"123456789000"
 
 Also, FWIW, I didn't censor anything here - most of the fields are
 just filled with generic data.
 
 --=20
 Rob Farmer

From: Pyun YongHyeon <pyunyh@gmail.com>
To: Rob Farmer <rfarmer@predatorlabs.net>
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Sat, 13 Nov 2010 19:17:25 -0800

 --0016367b64524671800494fac321
 Content-Type: text/plain; charset=ISO-8859-1
 Content-Transfer-Encoding: quoted-printable
 
 On Sat, Nov 13, 2010 at 6:32 PM, Rob Farmer <rfarmer@predatorlabs.net> wrot=
 e:
 > On Sat, Nov 13, 2010 at 17:10, Pyun YongHyeon <pyunyh@gmail.com> wrote:
 >> Oops, sorry, I should have drunk a cup of coffee here. :-(
 >> Show me the output of "kenv | grep smbios"
 >
 > smbios.bios.reldate=3D"03/12/2008"
 > smbios.bios.vendor=3D"Phoenix Technologies, LTD"
 > smbios.bios.version=3D"ASUS P5N32-SLI PREMIUM ACPI BIOS Revision 1002"
 > smbios.chassis.maker=3D"Chassis Manufacture"
 > smbios.chassis.serial=3D"EVAL =A0 =A0 =A0 =A0 =A0"
 > smbios.chassis.tag=3D"123456789000"
 > smbios.chassis.version=3D"Chassis Version"
 > smbios.memory.enabled=3D"4194304"
 > smbios.planar.maker=3D"ASUSTeK Computer INC."
 > smbios.planar.product=3D"P5N32-SLI PREMIUM"
 > smbios.planar.serial=3D"123456789000"
 > smbios.planar.version=3D"1.XX =A0 =A0"
 > smbios.socket.enabled=3D"1"
 > smbios.socket.populated=3D"1"
 > smbios.system.maker=3D"System manufacturer"
 > smbios.system.product=3D"System Product Name"
 > smbios.system.serial=3D"System Serial Number"
 > smbios.system.uuid=3D"10079fec-837a-da11-9e12-977b2bdc6f37"
 > smbios.system.version=3D"System Version"
 > smbios.version=3D"2.4"
 >
 >> Thanks for the output. Does hdac(4) works with MSI?
 >> I think you can check the output of "vmstat -i" and see whether
 >> the interrupt number for hdac(4) is 256 or higher.
 >
 > With MSI:
 > interrupt =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0total =A0 =
 =A0 =A0 rate
 > irq14: ata0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 2606 =A0 =A0 =
 =A0 =A0 =A07
 > irq16: vgapci0+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0629 =A0 =A0 =
 =A0 =A0 =A01
 > irq20: nve0 atapci2 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A094025 =A0 =A0 =A0 =A02=
 73
 > irq22: ohci0+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 2883 =A0 =A0 =
 =A0 =A0 =A08
 > irq23: hdac0 ehci0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 18352 =A0 =A0 =A0 =A0 =
 53
 > cpu0:timer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0123602 =A0 =A0 =
 =A0 =A0359
 > cpu1:timer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 79227 =A0 =A0 =
 =A0 =A0230
 > Total =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 321324 =A0 =
 =A0 =A0 =A0934
 >
 > Without MSI:
 > interrupt =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0total =A0 =
 =A0 =A0 rate
 > irq14: ata0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 1358 =A0 =A0 =
 =A0 =A0 =A07
 > irq16: vgapci0+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0317 =A0 =A0 =
 =A0 =A0 =A01
 > irq20: nfe0 atapci2 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A019971 =A0 =A0 =A0 =A01=
 07
 > irq22: ohci0+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 1937 =A0 =A0 =
 =A0 =A0 10
 > irq23: hdac0 ehci0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 13076 =A0 =A0 =A0 =A0 =
 70
 > cpu0:timer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 71326 =A0 =A0 =
 =A0 =A0383
 > cpu1:timer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 62319 =A0 =A0 =
 =A0 =A0335
 > Total =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 170304 =A0 =
 =A0 =A0 =A0915
 >
 > Works fine in both cases.
 >
 
 Thanks for testing. Would you try attached patch and let me
 know how it works? Note, you should not manually disable
 MSI/MSI-X before trying that patch. The patch should automatically
 detects your board and disable MSI/MSI-X.
 
 --0016367b64524671800494fac321
 Content-Type: application/octet-stream; name="nfe.msi.black.diff"
 Content-Disposition: attachment; filename="nfe.msi.black.diff"
 Content-Transfer-Encoding: base64
 X-Attachment-Id: f_gghctlep0
 
 SW5kZXg6IHN5cy9kZXYvbmZlL2lmX25mZS5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09
 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9kZXYvbmZlL2lm
 X25mZS5jCShyZXZpc2lvbiAyMTUyNzgpCisrKyBzeXMvZGV2L25mZS9pZl9uZmUuYwkod29ya2lu
 ZyBjb3B5KQpAQCAtNzcsNiArNzcsNyBAQAogc3RhdGljIGludCAgbmZlX3N1c3BlbmQoZGV2aWNl
 X3QpOwogc3RhdGljIGludCAgbmZlX3Jlc3VtZShkZXZpY2VfdCk7CiBzdGF0aWMgaW50IG5mZV9z
 aHV0ZG93bihkZXZpY2VfdCk7CitzdGF0aWMgaW50ICBuZmVfY2FuX3VzZV9tc2l4KHN0cnVjdCBu
 ZmVfc29mdGMgKik7CiBzdGF0aWMgdm9pZCBuZmVfcG93ZXIoc3RydWN0IG5mZV9zb2Z0YyAqKTsK
 IHN0YXRpYyBpbnQgIG5mZV9taWlidXNfcmVhZHJlZyhkZXZpY2VfdCwgaW50LCBpbnQpOwogc3Rh
 dGljIGludCAgbmZlX21paWJ1c193cml0ZXJlZyhkZXZpY2VfdCwgaW50LCBpbnQsIGludCk7CkBA
 IC0zODMsNiArMzg0LDEzIEBACiAJCQkgICAgIm1heC4gd2lkdGggb2YgbGluayh4JWQpXG4iLCB3
 aWR0aCwgdik7CiAJfQogCisJaWYgKG5mZV9jYW5fdXNlX21zaXgoc2MpID09IDApIHsKKwkJZGV2
 aWNlX3ByaW50ZihzYy0+bmZlX2RldiwKKwkJICAgICJNU0kvTVNJLVggY2FwYWJpbGl0eSBibGFj
 ay1saXN0ZWQsIHdpbGwgdXNlIElOVHhcbiIpOyAKKwkJbXNpeF9kaXNhYmxlID0gMTsKKwkJbXNp
 X2Rpc2FibGUgPSAxOworCX0KKwogCS8qIEFsbG9jYXRlIGludGVycnVwdCAqLwogCWlmIChtc2l4
 X2Rpc2FibGUgPT0gMCB8fCBtc2lfZGlzYWJsZSA9PSAwKSB7CiAJCWlmIChtc2l4X2Rpc2FibGUg
 PT0gMCAmJgpAQCAtNzgzLDYgKzc5MSw0MSBAQAogfQogCiAKK3N0YXRpYyBpbnQKK25mZV9jYW5f
 dXNlX21zaXgoc3RydWN0IG5mZV9zb2Z0YyAqc2MpCit7CisJc3RhdGljIHN0cnVjdCBtc2l4X2Js
 YWNrbGlzdCB7CisJCWNoYXIJKm1ha2VyOworCQljaGFyCSpwcm9kdWN0OworCX0gbXNpeF9ibGFj
 a2xpc3RzW10gPSB7CisJCXsgIkFTVVNUZUsgQ29tcHV0ZXIgSU5DLiIsICJQNU4zMi1TTEkgUFJF
 TUlVTSIgfQorCX07CisKKwlzdHJ1Y3QgbXNpeF9ibGFja2xpc3QgKm1ibHA7CisJY2hhciAqbWFr
 ZXIsICpwcm9kdWN0OworCWludCBjb3VudCwgbjsKKworCS8qCisJICogU2VhcmNoIGJhc2UgYm9h
 cmQgbWFudWZhY3R1cmVyIGFuZCBwcm9kdWN0IG5hbWUgdGFibGUKKwkgKiB0byBzZWUgdGhpcyBz
 eXN0ZW0gaGFzIGEga25vd24gTVNJL01TSS1YIGlzc3Vlcy4KKwkgKi8KKwltYWtlciA9IGdldGVu
 digic21iaW9zLnBsYW5hci5tYWtlciIpOworCXByb2R1Y3QgPSBnZXRlbnYoInNtYmlvcy5wbGFu
 YXIucHJvZHVjdCIpOworCWlmIChtYWtlciAhPSBOVUxMICYmIHByb2R1Y3QgIT0gTlVMTCkgewor
 CQljb3VudCA9IHNpemVvZihtc2l4X2JsYWNrbGlzdHMpIC8gc2l6ZW9mKG1zaXhfYmxhY2tsaXN0
 c1swXSk7CisJCW1ibHAgPSBtc2l4X2JsYWNrbGlzdHM7CisJCWZvciAobiA9IDA7IG4gPCBjb3Vu
 dDsgbisrKSB7CisJCQlpZiAoc3RyY21wKG1ha2VyLCBtYmxwLT5tYWtlcikgPT0gMCAmJgorCQkJ
 ICAgIHN0cmNtcChwcm9kdWN0LCBtYmxwLT5wcm9kdWN0KSA9PSAwKQorCQkJCXJldHVybiAoMCk7
 CisJCQltYmxwKys7CisJCX0KKwl9CisKKwlyZXR1cm4gKDEpOworfQorCisKIC8qIFRha2UgUEhZ
 L05JQyBvdXQgb2YgcG93ZXJkb3duLCBmcm9tIExpbnV4ICovCiBzdGF0aWMgdm9pZAogbmZlX3Bv
 d2VyKHN0cnVjdCBuZmVfc29mdGMgKnNjKQo=
 --0016367b64524671800494fac321--

From: Rob Farmer <rfarmer@predatorlabs.net>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152150: nfe(4) not working on Asus P5N32-SLI Premium
 motherboard (nforce 590)
Date: Sun, 14 Nov 2010 11:47:23 -0800

 On Sat, Nov 13, 2010 at 19:17, Pyun YongHyeon <pyunyh@gmail.com> wrote:
 > Thanks for testing. Would you try attached patch and let me
 > know how it works? Note, you should not manually disable
 > MSI/MSI-X before trying that patch. The patch should automatically
 > detects your board and disable MSI/MSI-X.
 >
 
 I removed the loader.conf stuff (so MSI and MSI-X are enabled again)
 and the patch works.
 
 Thanks for helping with this!
 -- 
 Rob Farmer

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/152150: commit references a PR
Date: Sun, 14 Nov 2010 23:37:47 +0000 (UTC)

 Author: yongari
 Date: Sun Nov 14 23:37:43 2010
 New Revision: 215327
 URL: http://svn.freebsd.org/changeset/base/215327
 
 Log:
   P5N32-SLI PREMIUM from ASUSTeK is known to have MSI/MSI-X issue
   such that nfe(4) does not work with MSI-X. When MSI-X support was
   introduced, I remember MCP55 controller worked without problems so
   the issue could be either PCI bridge or BIOS issue. But I also
   noticed snd_hda(4) disabled MSI on all MCP55 chipset so I'm still
   not sure this is generic issue of MCP55 chipset. If this was PCI
   bridge issue we would have added it to a system wide black-list
   table but it's not clear to me at this moment whether it was caused
   by either broken BIOS or silicon bug of MCP55 chipset.
   
   To workaround the issue, maintain a MSI/MSI-X black-list table in
   driver and lookup base board manufacturer and product name from the
   table before attempting to use MSI-X. If driver find an matching
   entry, nfe(4) will not use MSI/MSI-X and fall back on traditional
   INTx mode. This approach should be the last resort since it relies
   on smbios and if another instance of MSI/MSI-X breakage is reported
   with different maker/product, we may have to get the PCI bridge
   black-listed instead of adding an new entry.
   
   PR:	kern/152150
 
 Modified:
   head/sys/dev/nfe/if_nfe.c
 
 Modified: head/sys/dev/nfe/if_nfe.c
 ==============================================================================
 --- head/sys/dev/nfe/if_nfe.c	Sun Nov 14 23:05:57 2010	(r215326)
 +++ head/sys/dev/nfe/if_nfe.c	Sun Nov 14 23:37:43 2010	(r215327)
 @@ -77,6 +77,7 @@ static int  nfe_detach(device_t);
  static int  nfe_suspend(device_t);
  static int  nfe_resume(device_t);
  static int nfe_shutdown(device_t);
 +static int  nfe_can_use_msix(struct nfe_softc *);
  static void nfe_power(struct nfe_softc *);
  static int  nfe_miibus_readreg(device_t, int, int);
  static int  nfe_miibus_writereg(device_t, int, int, int);
 @@ -383,6 +384,13 @@ nfe_attach(device_t dev)
  			    "max. width of link(x%d)\n", width, v);
  	}
  
 +	if (nfe_can_use_msix(sc) == 0) {
 +		device_printf(sc->nfe_dev,
 +		    "MSI/MSI-X capability black-listed, will use INTx\n"); 
 +		msix_disable = 1;
 +		msi_disable = 1;
 +	}
 +
  	/* Allocate interrupt */
  	if (msix_disable == 0 || msi_disable == 0) {
  		if (msix_disable == 0 &&
 @@ -784,6 +792,41 @@ nfe_resume(device_t dev)
  }
  
  
 +static int
 +nfe_can_use_msix(struct nfe_softc *sc)
 +{
 +	static struct msix_blacklist {
 +		char	*maker;
 +		char	*product;
 +	} msix_blacklists[] = {
 +		{ "ASUSTeK Computer INC.", "P5N32-SLI PREMIUM" }
 +	};
 +
 +	struct msix_blacklist *mblp;
 +	char *maker, *product;
 +	int count, n;
 +
 +	/*
 +	 * Search base board manufacturer and product name table
 +	 * to see this system has a known MSI/MSI-X issue.
 +	 */
 +	maker = getenv("smbios.planar.maker");
 +	product = getenv("smbios.planar.product");
 +	if (maker != NULL && product != NULL) {
 +		count = sizeof(msix_blacklists) / sizeof(msix_blacklists[0]);
 +		mblp = msix_blacklists;
 +		for (n = 0; n < count; n++) {
 +			if (strcmp(maker, mblp->maker) == 0 &&
 +			    strcmp(product, mblp->product) == 0)
 +				return (0);
 +			mblp++;
 +		}
 +	}
 +
 +	return (1);
 +}
 +
 +
  /* Take PHY/NIC out of powerdown, from Linux */
  static void
  nfe_power(struct nfe_softc *sc)
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: feedback->patched 
State-Changed-By: yongari 
State-Changed-When: Mon Nov 15 00:32:38 UTC 2010 
State-Changed-Why:  
Thanks a lot for testing. Fixed in r215327. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152150 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/152150: commit references a PR
Date: Tue, 23 Nov 2010 19:11:35 +0000 (UTC)

 Author: yongari
 Date: Tue Nov 23 19:11:27 2010
 New Revision: 215759
 URL: http://svn.freebsd.org/changeset/base/215759
 
 Log:
   MFC r215327,215350:
   r215327:
     P5N32-SLI PREMIUM from ASUSTeK is known to have MSI/MSI-X issue
     such that nfe(4) does not work with MSI-X. When MSI-X support was
     introduced, I remember MCP55 controller worked without problems so
     the issue could be either PCI bridge or BIOS issue. But I also
     noticed snd_hda(4) disabled MSI on all MCP55 chipset so I'm still
     not sure this is generic issue of MCP55 chipset. If this was PCI
     bridge issue we would have added it to a system wide black-list
     table but it's not clear to me at this moment whether it was caused
     by either broken BIOS or silicon bug of MCP55 chipset.
   
     To workaround the issue, maintain a MSI/MSI-X black-list table in
     driver and lookup base board manufacturer and product name from the
     table before attempting to use MSI-X. If driver find an matching
     entry, nfe(4) will not use MSI/MSI-X and fall back on traditional
     INTx mode. This approach should be the last resort since it relies
     on smbios and if another instance of MSI/MSI-X breakage is reported
     with different maker/product, we may have to get the PCI bridge
     black-listed instead of adding an new entry.
   
     PR:	kern/152150
   
   r215350:
     Plug memory leakage introduced in r215327.
   
     Submitted by:	jkim
 
 Modified:
   stable/8/sys/dev/nfe/if_nfe.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
   stable/8/sys/dev/xen/xenpci/   (props changed)
 
 Modified: stable/8/sys/dev/nfe/if_nfe.c
 ==============================================================================
 --- stable/8/sys/dev/nfe/if_nfe.c	Tue Nov 23 18:52:27 2010	(r215758)
 +++ stable/8/sys/dev/nfe/if_nfe.c	Tue Nov 23 19:11:27 2010	(r215759)
 @@ -77,6 +77,7 @@ static int  nfe_detach(device_t);
  static int  nfe_suspend(device_t);
  static int  nfe_resume(device_t);
  static int nfe_shutdown(device_t);
 +static int  nfe_can_use_msix(struct nfe_softc *);
  static void nfe_power(struct nfe_softc *);
  static int  nfe_miibus_readreg(device_t, int, int);
  static int  nfe_miibus_writereg(device_t, int, int, int);
 @@ -383,6 +384,13 @@ nfe_attach(device_t dev)
  			    "max. width of link(x%d)\n", width, v);
  	}
  
 +	if (nfe_can_use_msix(sc) == 0) {
 +		device_printf(sc->nfe_dev,
 +		    "MSI/MSI-X capability black-listed, will use INTx\n"); 
 +		msix_disable = 1;
 +		msi_disable = 1;
 +	}
 +
  	/* Allocate interrupt */
  	if (msix_disable == 0 || msi_disable == 0) {
  		if (msix_disable == 0 &&
 @@ -783,6 +791,48 @@ nfe_resume(device_t dev)
  }
  
  
 +static int
 +nfe_can_use_msix(struct nfe_softc *sc)
 +{
 +	static struct msix_blacklist {
 +		char	*maker;
 +		char	*product;
 +	} msix_blacklists[] = {
 +		{ "ASUSTeK Computer INC.", "P5N32-SLI PREMIUM" }
 +	};
 +
 +	struct msix_blacklist *mblp;
 +	char *maker, *product;
 +	int count, n, use_msix;
 +
 +	/*
 +	 * Search base board manufacturer and product name table
 +	 * to see this system has a known MSI/MSI-X issue.
 +	 */
 +	maker = getenv("smbios.planar.maker");
 +	product = getenv("smbios.planar.product");
 +	use_msix = 1;
 +	if (maker != NULL && product != NULL) {
 +		count = sizeof(msix_blacklists) / sizeof(msix_blacklists[0]);
 +		mblp = msix_blacklists;
 +		for (n = 0; n < count; n++) {
 +			if (strcmp(maker, mblp->maker) == 0 &&
 +			    strcmp(product, mblp->product) == 0) {
 +				use_msix = 0;
 +				break;
 +			}
 +			mblp++;
 +		}
 +	}
 +	if (maker != NULL)
 +		freeenv(maker);
 +	if (product != NULL)
 +		freeenv(product);
 +
 +	return (use_msix);
 +}
 +
 +
  /* Take PHY/NIC out of powerdown, from Linux */
  static void
  nfe_power(struct nfe_softc *sc)
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/152150: commit references a PR
Date: Tue, 23 Nov 2010 19:14:25 +0000 (UTC)

 Author: yongari
 Date: Tue Nov 23 19:14:17 2010
 New Revision: 215760
 URL: http://svn.freebsd.org/changeset/base/215760
 
 Log:
   MFC r215327,215350:
   r215327:
     P5N32-SLI PREMIUM from ASUSTeK is known to have MSI/MSI-X issue
     such that nfe(4) does not work with MSI-X. When MSI-X support was
     introduced, I remember MCP55 controller worked without problems so
     the issue could be either PCI bridge or BIOS issue. But I also
     noticed snd_hda(4) disabled MSI on all MCP55 chipset so I'm still
     not sure this is generic issue of MCP55 chipset. If this was PCI
     bridge issue we would have added it to a system wide black-list
     table but it's not clear to me at this moment whether it was caused
     by either broken BIOS or silicon bug of MCP55 chipset.
   
     To workaround the issue, maintain a MSI/MSI-X black-list table in
     driver and lookup base board manufacturer and product name from the
     table before attempting to use MSI-X. If driver find an matching
     entry, nfe(4) will not use MSI/MSI-X and fall back on traditional
     INTx mode. This approach should be the last resort since it relies
     on smbios and if another instance of MSI/MSI-X breakage is reported
     with different maker/product, we may have to get the PCI bridge
     black-listed instead of adding an new entry.
   
     PR:	kern/152150
   
   r215350:
     Plug memory leakage introduced in r215327.
   
     Submitted by:	jkim
 
 Modified:
   stable/7/sys/dev/nfe/if_nfe.c
 Directory Properties:
   stable/7/sys/   (props changed)
   stable/7/sys/cddl/contrib/opensolaris/   (props changed)
   stable/7/sys/contrib/dev/acpica/   (props changed)
   stable/7/sys/contrib/pf/   (props changed)
 
 Modified: stable/7/sys/dev/nfe/if_nfe.c
 ==============================================================================
 --- stable/7/sys/dev/nfe/if_nfe.c	Tue Nov 23 19:11:27 2010	(r215759)
 +++ stable/7/sys/dev/nfe/if_nfe.c	Tue Nov 23 19:14:17 2010	(r215760)
 @@ -77,6 +77,7 @@ static int  nfe_detach(device_t);
  static int  nfe_suspend(device_t);
  static int  nfe_resume(device_t);
  static int nfe_shutdown(device_t);
 +static int  nfe_can_use_msix(struct nfe_softc *);
  static void nfe_power(struct nfe_softc *);
  static int  nfe_miibus_readreg(device_t, int, int);
  static int  nfe_miibus_writereg(device_t, int, int, int);
 @@ -383,6 +384,13 @@ nfe_attach(device_t dev)
  			    "max. width of link(x%d)\n", width, v);
  	}
  
 +	if (nfe_can_use_msix(sc) == 0) {
 +		device_printf(sc->nfe_dev,
 +		    "MSI/MSI-X capability black-listed, will use INTx\n"); 
 +		msix_disable = 1;
 +		msi_disable = 1;
 +	}
 +
  	/* Allocate interrupt */
  	if (msix_disable == 0 || msi_disable == 0) {
  		if (msix_disable == 0 &&
 @@ -784,6 +792,48 @@ nfe_resume(device_t dev)
  }
  
  
 +static int
 +nfe_can_use_msix(struct nfe_softc *sc)
 +{
 +	static struct msix_blacklist {
 +		char	*maker;
 +		char	*product;
 +	} msix_blacklists[] = {
 +		{ "ASUSTeK Computer INC.", "P5N32-SLI PREMIUM" }
 +	};
 +
 +	struct msix_blacklist *mblp;
 +	char *maker, *product;
 +	int count, n, use_msix;
 +
 +	/*
 +	 * Search base board manufacturer and product name table
 +	 * to see this system has a known MSI/MSI-X issue.
 +	 */
 +	maker = getenv("smbios.planar.maker");
 +	product = getenv("smbios.planar.product");
 +	use_msix = 1;
 +	if (maker != NULL && product != NULL) {
 +		count = sizeof(msix_blacklists) / sizeof(msix_blacklists[0]);
 +		mblp = msix_blacklists;
 +		for (n = 0; n < count; n++) {
 +			if (strcmp(maker, mblp->maker) == 0 &&
 +			    strcmp(product, mblp->product) == 0) {
 +				use_msix = 0;
 +				break;
 +			}
 +			mblp++;
 +		}
 +	}
 +	if (maker != NULL)
 +		freeenv(maker);
 +	if (product != NULL)
 +		freeenv(product);
 +
 +	return (use_msix);
 +}
 +
 +
  /* Take PHY/NIC out of powerdown, from Linux */
  static void
  nfe_power(struct nfe_softc *sc)
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: yongari 
State-Changed-When: Tue Nov 23 19:38:50 UTC 2010 
State-Changed-Why:  
MFC to stable/8 and stable/7 completed. 
Thanks a lot for testing and reporting! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152150 
>Unformatted:
 >>>ifconfig with nfe:
 
 nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=8019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,LINKSTATE>
 	ether 00:18:f3:af:e0:9c
 	inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
 	media: Ethernet autoselect (1000baseT <full-duplex,flag0,flag1>)
 	status: active
 nfe1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=8019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,LINKSTATE>
 	ether 00:18:f3:af:e7:42
 	media: Ethernet autoselect (none)
 	status: no carrier
 pflog0: flags=0<> metric 0 mtu 33152
 lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
 	options=3<RXCSUM,TXCSUM>
 	inet 127.0.0.1 netmask 0xff000000 
 	inet6 ::1 prefixlen 128 
 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 pfsync0: flags=0<> metric 0 mtu 1460
 	syncpeer: 224.0.0.240 maxupd: 128
 
 >>>ifconfig with nve:
 
 nve0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=80008<VLAN_MTU,LINKSTATE>
 	ether 00:18:f3:af:e0:9c
 	inet 128.95.133.119 netmask 0xffffff00 broadcast 128.95.133.255
 	media: Ethernet autoselect (1000baseT <full-duplex,flag0,flag1>)
 	status: active
 nve1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=80008<VLAN_MTU,LINKSTATE>
 	ether 00:18:f3:af:e7:42
 	media: Ethernet autoselect (none)
 	status: no carrier
 lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
 	options=3<RXCSUM,TXCSUM>
 	inet 127.0.0.1 netmask 0xff000000 
 	inet6 ::1 prefixlen 128 
 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 pfsync0: flags=0<> metric 0 mtu 1460
 	syncpeer: 224.0.0.240 maxupd: 128
 pflog0: flags=0<> metric 0 mtu 33152
 
 >>>pciconf -lv:
 
 hostb0@pci0:0:0:0:	class=0x060000 card=0x00000000 chip=0x007110de rev=0xc1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition CPU to PCI Bridge'
     class      = bridge
     subclass   = HOST-PCI
 none0@pci0:0:0:1:	class=0x050000 card=0x00000000 chip=0x007f10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none1@pci0:0:0:2:	class=0x050000 card=0x00000000 chip=0x007510de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none2@pci0:0:0:3:	class=0x050000 card=0x00000000 chip=0x006f10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none3@pci0:0:0:4:	class=0x050000 card=0x00000000 chip=0x00b410de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none4@pci0:0:1:0:	class=0x050000 card=0x00000000 chip=0x007610de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none5@pci0:0:1:1:	class=0x050000 card=0x00000000 chip=0x007810de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none6@pci0:0:1:2:	class=0x050000 card=0x00000000 chip=0x007910de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none7@pci0:0:1:3:	class=0x050000 card=0x00000000 chip=0x007a10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none8@pci0:0:1:4:	class=0x050000 card=0x00000000 chip=0x007b10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none9@pci0:0:1:5:	class=0x050000 card=0x00000000 chip=0x007c10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 none10@pci0:0:1:6:	class=0x050000 card=0x00000000 chip=0x007d10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition Memory Registers'
     class      = memory
     subclass   = RAM
 pcib1@pci0:0:2:0:	class=0x060400 card=0x00000000 chip=0x007e10de rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition PCIe Root Hub'
     class      = bridge
     subclass   = PCI-PCI
 pcib2@pci0:0:5:0:	class=0x060400 card=0x00000000 chip=0x007e10de rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'nForce4 Intel Edition PCIe Root Hub'
     class      = bridge
     subclass   = PCI-PCI
 none11@pci0:0:9:0:	class=0x050000 card=0xcb841043 chip=0x036910de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Memory Controller'
     class      = memory
     subclass   = RAM
 isab0@pci0:0:10:0:	class=0x060100 card=0xcb841043 chip=0x036010de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 LPC Bridge'
     class      = bridge
     subclass   = PCI-ISA
 none12@pci0:0:10:1:	class=0x0c0500 card=0xcb841043 chip=0x036810de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'SMBus controller ((0xCB84 integrated chip nForce Pro 3400))'
     class      = serial bus
     subclass   = SMBus
 none13@pci0:0:10:2:	class=0x050000 card=0xcb841043 chip=0x036a10de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Memory Controller'
     class      = memory
     subclass   = RAM
 ohci0@pci0:0:11:0:	class=0x0c0310 card=0xcb841043 chip=0x036c10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'Standard OpenHCD USB Hostcontroller (81FB1043)'
     class      = serial bus
     subclass   = USB
 ehci0@pci0:0:11:1:	class=0x0c0320 card=0xcb841043 chip=0x036d10de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'Standard PCI-to-USB Enhanced Hostcontroller (81FB1043)'
     class      = serial bus
     subclass   = USB
 atapci1@pci0:0:13:0:	class=0x01018a card=0xcb841043 chip=0x036e10de rev=0xa1 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PATA Controller (MCP55)'
     class      = mass storage
     subclass   = ATA
 atapci2@pci0:0:14:0:	class=0x010185 card=0xcb841043 chip=0x037f10de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 SATA/RAID Controller (MCP55S)'
     class      = mass storage
     subclass   = ATA
 atapci3@pci0:0:14:1:	class=0x010185 card=0xcb841043 chip=0x037f10de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 SATA/RAID Controller (MCP55S)'
     class      = mass storage
     subclass   = ATA
 atapci4@pci0:0:14:2:	class=0x010185 card=0xcb841043 chip=0x037f10de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 SATA/RAID Controller (MCP55S)'
     class      = mass storage
     subclass   = ATA
 pcib3@pci0:0:15:0:	class=0x060401 card=0xcb8410de chip=0x037010de rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCI bridge'
     class      = bridge
     subclass   = PCI-PCI
 hdac0@pci0:0:15:1:	class=0x040300 card=0x81f61043 chip=0x037110de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'High Definition Audio Controller (MCP55)'
     class      = multimedia
     subclass   = HDA
 nve0@pci0:0:17:0:	class=0x068000 card=0xcb841043 chip=0x037310de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Ethernet'
     class      = bridge
 nve1@pci0:0:18:0:	class=0x068000 card=0xcb841043 chip=0x037310de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 Ethernet'
     class      = bridge
 pcib4@pci0:0:19:0:	class=0x060400 card=0x000010de chip=0x037610de rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCIe bridge'
     class      = bridge
     subclass   = PCI-PCI
 pcib5@pci0:0:23:0:	class=0x060400 card=0x000010de chip=0x037510de rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCIe bridge'
     class      = bridge
     subclass   = PCI-PCI
 pcib6@pci0:0:24:0:	class=0x060400 card=0x000010de chip=0x037710de rev=0xa2 hdr=0x01
     vendor     = 'NVIDIA Corporation'
     device     = 'MCP55 PCIe bridge'
     class      = bridge
     subclass   = PCI-PCI
 vgapci0@pci0:1:0:0:	class=0x030000 card=0x042010de chip=0x019310de rev=0xa2 hdr=0x00
     vendor     = 'NVIDIA Corporation'
     device     = 'NVIDIA GeForce 8800 GTS (G80)'
     class      = display
     subclass   = VGA
 atapci0@pci0:2:0:0:	class=0x018000 card=0x819f1043 chip=0x31321095 rev=0x01 hdr=0x00
     vendor     = 'Silicon Image Inc (Was: CMD Technology Inc)'
     device     = 'PCI Express (1x) to 2 Port SATA300 (SiI 3132)'
     class      = mass storage
 none14@pci0:3:7:0:	class=0x040000 card=0x74440070 chip=0x5b7a14f1 rev=0x00 hdr=0x00
     vendor     = 'Conexant Systems, Inc.'
     device     = 'Single-Chip MPEG-2 Encoder with Integrated Analog Video/Broadcast Audio Decoder (Belived to be a CX23416)'
     class      = multimedia
     subclass   = video
 
 >>>dmesg with nfe:
 
 Copyright (c) 1992-2010 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 	The Regents of the University of California. All rights reserved.
 FreeBSD is a registered trademark of The FreeBSD Foundation.
 FreeBSD 9.0-CURRENT #1 r215105: Thu Nov 11 11:31:16 PST 2010
     rfarmer@amethyst.predatorlabs.net:/usr/obj/usr/src/sys/AMETHYST amd64
 WARNING: WITNESS option enabled, expect reduced performance.
 CPU: Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz (2400.20-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x6f6  Family = 6  Model = f  Stepping = 6
   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
   Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
   AMD Features=0x20100800<SYSCALL,NX,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant
 real memory  = 4294967296 (4096 MB)
 avail memory = 4080021504 (3891 MB)
 Event timer "LAPIC" quality 400
 ACPI APIC Table: <Nvidia ASUSACPI>
 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 FreeBSD/SMP: 1 package(s) x 2 core(s)
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
 ioapic0: Changing APIC ID to 4
 ioapic0 <Version 1.1> irqs 0-23 on motherboard
 kbd1 at kbdmux0
 acpi0: <Nvidia ASUSACPI> on motherboard
 acpi0: Power Button (fixed)
 acpi0: reservation of 0, a0000 (3) failed
 acpi0: reservation of 100000, bfdf0000 (3) failed
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 acpi_button0: <Power Button> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 pci0: <memory, RAM> at device 0.1 (no driver attached)
 pci0: <memory, RAM> at device 0.2 (no driver attached)
 pci0: <memory, RAM> at device 0.3 (no driver attached)
 pci0: <memory, RAM> at device 0.4 (no driver attached)
 pci0: <memory, RAM> at device 1.0 (no driver attached)
 pci0: <memory, RAM> at device 1.1 (no driver attached)
 pci0: <memory, RAM> at device 1.2 (no driver attached)
 pci0: <memory, RAM> at device 1.3 (no driver attached)
 pci0: <memory, RAM> at device 1.4 (no driver attached)
 pci0: <memory, RAM> at device 1.5 (no driver attached)
 pci0: <memory, RAM> at device 1.6 (no driver attached)
 pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 vgapci0: <VGA-compatible display> port 0x8c00-0x8c7f mem 0xfa000000-0xfaffffff,0xd0000000-0xdfffffff,0xf8000000-0xf9ffffff irq 16 at device 0.0 on pci1
 nvidia0: <GeForce 8800 GTS> on vgapci0
 vgapci0: child nvidia0 requested pci_enable_busmaster
 vgapci0: child nvidia0 requested pci_enable_io
 vgapci0: child nvidia0 requested pci_enable_io
 pcib2: <ACPI PCI-PCI bridge> at device 5.0 on pci0
 pci2: <ACPI PCI bus> on pcib2
 atapci0: <SiI 3132 SATA300 controller> port 0x9c00-0x9c7f mem 0xfdfff000-0xfdfff07f,0xfdff8000-0xfdffbfff irq 16 at device 0.0 on pci2
 ata2: <ATA channel 0> on atapci0
 ata3: <ATA channel 1> on atapci0
 pci0: <memory, RAM> at device 9.0 (no driver attached)
 isab0: <PCI-ISA bridge> at device 10.0 on pci0
 isa0: <ISA bus> on isab0
 pci0: <serial bus, SMBus> at device 10.1 (no driver attached)
 pci0: <memory, RAM> at device 10.2 (no driver attached)
 ohci0: <nVidia nForce MCP55 USB Controller> mem 0xfe02f000-0xfe02ffff at device 11.0 on pci0
 usbus0: <nVidia nForce MCP55 USB Controller> on ohci0
 ehci0: <NVIDIA nForce MCP55 USB 2.0 controller> mem 0xfe02e000-0xfe02e0ff at device 11.1 on pci0
 usbus1: EHCI version 1.0
 usbus1: <NVIDIA nForce MCP55 USB 2.0 controller> on ehci0
 atapci1: <nVidia nForce MCP55 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 13.0 on pci0
 ata0: <ATA channel 0> on atapci1
 ata1: <ATA channel 1> on atapci1
 atapci2: <nVidia nForce MCP55 SATA300 controller> port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xdc00-0xdc0f mem 0xfe02d000-0xfe02dfff irq 20 at device 14.0 on pci0
 ata4: <ATA channel 0> on atapci2
 ata5: <ATA channel 1> on atapci2
 atapci3: <nVidia nForce MCP55 SATA300 controller> port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xc800-0xc80f mem 0xfe02c000-0xfe02cfff irq 21 at device 14.1 on pci0
 ata6: <ATA channel 0> on atapci3
 ata7: <ATA channel 1> on atapci3
 atapci4: <nVidia nForce MCP55 SATA300 controller> port 0xc400-0xc407,0xc000-0xc003,0xbc00-0xbc07,0xb800-0xb803,0xb400-0xb40f mem 0xfe02b000-0xfe02bfff irq 22 at device 14.2 on pci0
 ata8: <ATA channel 0> on atapci4
 ata9: <ATA channel 1> on atapci4
 pcib3: <ACPI PCI-PCI bridge> at device 15.0 on pci0
 pci3: <ACPI PCI bus> on pcib3
 pci3: <multimedia, video> at device 7.0 (no driver attached)
 hdac0: <NVidia MCP55 High Definition Audio Controller> mem 0xfe020000-0xfe023fff irq 23 at device 15.1 on pci0
 hdac0: HDA Driver Revision: 20100226_0142
 nfe0: <NVIDIA nForce MCP55 Networking Adapter> port 0xb000-0xb007 mem 0xfe02a000-0xfe02afff,0xfe029000-0xfe0290ff,0xfe028000-0xfe02800f irq 20 at device 17.0 on pci0
 miibus0: <MII bus> on nfe0
 e1000phy0: <Marvell 88E1116 Gigabit PHY> PHY 1 on miibus0
 e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
 nfe0: Ethernet address: 00:18:f3:af:e0:9c
 nfe1: <NVIDIA nForce MCP55 Networking Adapter> port 0xac00-0xac07 mem 0xfe027000-0xfe027fff,0xfe026000-0xfe0260ff,0xfe025000-0xfe02500f irq 21 at device 18.0 on pci0
 miibus1: <MII bus> on nfe1
 e1000phy1: <Marvell 88E1116 Gigabit PHY> PHY 1 on miibus1
 e1000phy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
 nfe1: Ethernet address: 00:18:f3:af:e7:42
 pcib4: <ACPI PCI-PCI bridge> at device 19.0 on pci0
 pci4: <ACPI PCI bus> on pcib4
 pcib5: <ACPI PCI-PCI bridge> at device 23.0 on pci0
 pci5: <ACPI PCI bus> on pcib5
 pcib6: <ACPI PCI-PCI bridge> at device 24.0 on pci0
 pci6: <ACPI PCI bus> on pcib6
 acpi_tz0: <Thermal Zone> on acpi0
 attimer0: <AT timer> port 0x40-0x43 on acpi0
 Timecounter "i8254" frequency 1193182 Hz quality 0
 Event timer "i8254" frequency 1193182 Hz quality 100
 hpet0: <High Precision Event Timer> iomem 0xfefff000-0xfefff3ff irq 0,8 on acpi0
 device_attach: hpet0 attach returned 12
 atrtc0: <AT realtime clock> port 0x70-0x73 on acpi0
 Event timer "RTC" frequency 32768 Hz quality 0
 orm0: <ISA Option ROM> at iomem 0xd0000-0xd3fff on isa0
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x300>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 coretemp0: <CPU On-Die Thermal Sensors> on cpu0
 est0: <Enhanced SpeedStep Frequency Control> on cpu0
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr 92a092a0600092a
 device_attach: est0 attach returned 6
 p4tcc0: <CPU Frequency Thermal Control> on cpu0
 coretemp1: <CPU On-Die Thermal Sensors> on cpu1
 est1: <Enhanced SpeedStep Frequency Control> on cpu1
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr 92a092a0600092a
 device_attach: est1 attach returned 6
 p4tcc1: <CPU Frequency Thermal Control> on cpu1
 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
             to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
 ZFS filesystem version 4
 ZFS storage pool version 15
 Timecounters tick every 1.000 msec
 usbus0: 12Mbps Full Speed USB v1.0
 usbus1: 480Mbps High Speed USB v2.0
 acd0: DVDROM <LITE-ON DVD SHD-16P1S/GS03> at ata0-master UDMA33 
 ad8: 476940MB <WDC WD5000AAKS-00UU3A0 01.03B01> at ata4-master UDMA100 SATA 3Gb/s
 ugen0.1: <nVidia> at usbus0
 uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
 ugen1.1: <nVidia> at usbus1
 uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
 ad10: 70911MB <WDC WD740ADFS-00SLR5 21.07QR5> at ata5-master UDMA100 SATA 3Gb/s
 hdac0: HDA Codec #0: Analog Devices AD1988B
 pcm0: <HDA Analog Devices AD1988B PCM #0 Analog> at cad 0 nid 1 on hdac0
 pcm1: <HDA Analog Devices AD1988B PCM #1 Analog> at cad 0 nid 1 on hdac0
 pcm2: <HDA Analog Devices AD1988B PCM #2 Digital> at cad 0 nid 1 on hdac0
 SMP: AP CPU #1 Launched!
 WARNING: WITNESS option enabled, expect reduced performance.
 GEOM: ad10: partition 2 does not start on a track boundary.
 GEOM: ad10: partition 2 does not end on a track boundary.
 GEOM: ad10: partition 1 does not start on a track boundary.
 GEOM: ad10: partition 1 does not end on a track boundary.
 Root mount waiting for: usbus1 usbus0
 uhub0: 10 ports with 10 removable, self powered
 Root mount waiting for: usbus1
 Root mount waiting for: usbus1
 Root mount waiting for: usbus1
 uhub1: 10 ports with 10 removable, self powered
 Root mount waiting for: usbus1
 Root mount waiting for: usbus1
 ugen0.2: <Logitech> at usbus0
 ums0: <Logitech USB-PS2 Optical Mouse, class 0/0, rev 2.00/11.10, addr 2> on usbus0
 ums0: 3 buttons and [XYZ] coordinates ID=0
 uhub_reattach_port: port 3 reset failed, error=USB_ERR_TIMEOUT
 uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 3
 Trying to mount root from zfs:tank []...
 ugen1.2: <vendor 0x0bda> at usbus1
 ugen0.3: <Generic> at usbus0
 ukbd0: <Generic USB+PS2 Keyboard, class 0/0, rev 1.10/2.02, addr 3> on usbus0
 kbd2 at ukbd0
 uhid0: <Generic USB+PS2 Keyboard, class 0/0, rev 1.10/2.02, addr 3> on usbus0
 uhub_reattach_port: port 4 reset failed, error=USB_ERR_TIMEOUT
 uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 4
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 lock order reversal:
  1st 0xffffff0002bca848 filedesc structure (filedesc structure) @ /usr/src/sys/kern/kern_descrip.c:1133
  2nd 0xffffff0135e5fdb8 zfs (zfs) @ /usr/src/sys/kern/vfs_subr.c:4176
 KDB: stack backtrace:
 db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
 kdb_backtrace() at kdb_backtrace+0x37
 _witness_debugger() at _witness_debugger+0x49
 witness_checkorder() at witness_checkorder+0x7b3
 __lockmgr_args() at __lockmgr_args+0xcd6
 vop_stdlock() at vop_stdlock+0x39
 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
 _vn_lock() at _vn_lock+0x50
 knlist_remove_kq() at knlist_remove_kq+0x67
 knote_fdclose() at knote_fdclose+0x169
 kern_close() at kern_close+0xc0
 syscallenter() at syscallenter+0x182
 syscall() at syscall+0x41
 Xfast_syscall() at Xfast_syscall+0xe2
 --- syscall (6, FreeBSD ELF64, close), rip = 0x8010b4ddc, rsp = 0x7fffffffe868, rbp = 0x80286a6d0 ---
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 nfe0: watchdog timeout (missed Tx interrupts) -- recovering
 
