From nobody@FreeBSD.org  Tue Nov 16 13:14:52 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 15287106564A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 16 Nov 2010 13:14:52 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 00F118FC08
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 16 Nov 2010 13:14:52 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id oAGDEpYv083898
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 16 Nov 2010 13:14:51 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id oAGDEpom083897;
	Tue, 16 Nov 2010 13:14:51 GMT
	(envelope-from nobody)
Message-Id: <201011161314.oAGDEpom083897@www.freebsd.org>
Date: Tue, 16 Nov 2010 13:14:51 GMT
From: Mykola Zubach <zuborg@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         152295
>Category:       kern
>Synopsis:       [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    yongari
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Nov 16 13:20:03 UTC 2010
>Closed-Date:    Wed Feb 02 01:07:30 UTC 2011
>Last-Modified:  Wed Feb 02 01:07:30 UTC 2011
>Originator:     Mykola Zubach
>Release:        8.1-RELEASE
>Organization:
AdvancedHosters.com
>Environment:
FreeBSD 23382 7.3-RELEASE-p2 FreeBSD 7.3-RELEASE-p2 #0: Fri Jul 16 14:22:06 UTC 2010     root@DS1105:/usr/obj/usr/src/sys/Z-AMD64  amd64
>Description:
On 8.1-RELEASE (both custom and GENERIC kernels) after some period of quite high bandwidth (10-20 minutes of 600-800Mbit/s) bge0 device fail to receive any incoming packets.

After such fail 'tcpdump' show no incoming traffic, ssh still provide some amount of outgoing data (like 'top' or 'nload' output), but keypresses don't reach the server and tcp session drops after timeout.

These two commands restore working state
# ifconfig bge0 down
# ifconfig bge0 up

Without 'polling' driver hangs up very quicky, in 1-2 minutes. Active 'polling' can provide 10-20 minutes of networking.

Bandwidth is produced by 'nginx' webserver. Various configurations were tested, like 'aio sendfile' feature, or pure read/write operations without 'sendfile on' - in all cases bge fails.

Kernel is compiled with ZERO_COPY_SOCKETS option, but GENERIC kernel fails too.

7.3-RELEASE works fine under same high load.

# dmesg
FreeBSD 8.1-RELEASE-p1 #0: Mon Oct 18 11:31:13 UTC 2010
    root@DS1728:/usr/obj/usr/src/sys/Z-AMD64 amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU           X3430  @ 2.40GHz (2394.00-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106e5  Family = 6  Model = 1e  Stepping = 5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x98e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 17179869184 (16384 MB)
avail memory = 16546603008 (15780 MB)
ACPI APIC Table: <HP     ProLiant>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  2
 cpu2 (AP): APIC ID:  4
 cpu3 (AP): APIC ID:  6
ioapic0 <Version 2.0> irqs 0-23 on motherboard
netisr_init: forcing maxthreads to 1 and bindthreads to 0 for device polling
kbd1 at kbdmux0
acpi0: <HP ProLiant> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 3.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pci0: <base peripheral> at device 8.0 (no driver attached)
pci0: <base peripheral> at device 8.1 (no driver attached)
pci0: <base peripheral> at device 8.2 (no driver attached)
pci0: <base peripheral> at device 8.3 (no driver attached)
pci0: <base peripheral> at device 16.0 (no driver attached)
pci0: <base peripheral> at device 16.1 (no driver attached)
ehci0: <Intel PCH USB 2.0 controller USB-B> mem 0xdfd02000-0xdfd023ff irq 16 at device 26.0 on pci0
ehci0: [ITHREAD]
usbus0: EHCI version 1.0
usbus0: <Intel PCH USB 2.0 controller USB-B> on ehci0
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci16: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0
pci32: <ACPI PCI bus> on pcib3
bge0: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem 0xdf900000-0xdf90ffff irq 16 at device 0.0 on pci32
miibus0: <MII bus> on bge0
brgphy0: <BCM5784 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge0: Ethernet address: d4:85:64:38:ff:70
bge0: [FILTER]
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.5 on pci0
pci34: <ACPI PCI bus> on pcib4
bge1: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem 0xdfa00000-0xdfa0ffff irq 17 at device 0.0 on pci34
miibus1: <MII bus> on bge1
brgphy1: <BCM5784 10/100/1000baseTX PHY> PHY 1 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
bge1: Ethernet address: d4:85:64:38:ff:71
bge1: [FILTER]
pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.6 on pci0
pci36: <ACPI PCI bus> on pcib5
vgapci0: <VGA-compatible display> mem 0xde000000-0xdeffffff,0xdf800000-0xdf803fff,0xdf000000-0xdf7fffff irq 18 at device 0.0 on pci36
pcib6: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0
pci38: <ACPI PCI bus> on pcib6
ehci1: <Intel PCH USB 2.0 controller USB-A> mem 0xdfd02400-0xdfd027ff irq 23 at device 29.0 on pci0
ehci1: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <Intel PCH USB 2.0 controller USB-A> on ehci1
pcib7: <PCI-PCI bridge> at device 30.0 on pci0
pci48: <PCI bus> on pcib7
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PCH SATA300 controller> port 0x1830-0x1837,0x1824-0x1827,0x1828-0x182f,0x1820-0x1823,0x1800-0x181f mem 0xdfd01000-0xdfd017ff irq 18 at device 31.2 on pci0
atapci0: [ITHREAD]
atapci0: AHCI called from vendor specific driver
atapci0: AHCI v1.30 controller with 6 3Gbps ports, PM supported
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
ata4: <ATA channel 2> on atapci0
ata4: [ITHREAD]
ata5: <ATA channel 3> on atapci0
ata5: [ITHREAD]
ata6: <ATA channel 4> on atapci0
ata6: [ITHREAD]
ata7: <ATA channel 5> on atapci0
ata7: [ITHREAD]
ichsmb0: <Intel PCH SMBus controller> port 0x1840-0x185f mem 0xdfd02800-0xdfd028ff irq 18 at device 31.3 on pci0
ichsmb0: [ITHREAD]
smbus0: <System Management Bus> on ichsmb0
smb0: <SMBus generic I/O> on smbus0
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdf000-0xdffff on isa0
atkbd: unable to set the command byte.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: unable to set the command byte.
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
Timecounter "TSC" frequency 2393999100 Hz quality 800
Timecounters tick every 0.976 msec
ipfw2 initialized, divert loadable, nat loadable, rule-based forwarding disabled, default to accept, logging disabled
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ad4: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata2-master UDMA100 SATA 3Gb/s
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ad6: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata3-master UDMA100 SATA 3Gb/s
GEOM: ad4s1: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad4s1: media size does not match label.
ad8: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata4-master UDMA100 SATA 3Gb/s
GEOM: ad4s2: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad4s2: media size does not match label.
ad10: 953869MB <WDC WD1002FBYS-18W8B1 03.00C10> at ata5-master UDMA100 SATA 3Gb/s
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
GEOM: ad6s1: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad6s1: media size does not match label.
GEOM: ad6s2: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad6s2: media size does not match label.
GEOM: ad8s1: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad8s1: media size does not match label.
GEOM_MIRROR: Device mirror/gm0 launched (3/3).
GEOM: ad8s2: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad8s2: media size does not match label.
GEOM: ad10s1: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad10s1: media size does not match label.
GEOM_MIRROR: Device mirror/gm2 launched (1/1).
GEOM: ad10s2: geometry does not match label (255h,63s != 16h,63s).
GEOM: ad10s2: media size does not match label.
GEOM_MIRROR: Device mirror/gm1 launched (4/4).
Root mount waiting for: usbus1 usbus0
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
Root mount waiting for: usbus1 usbus0
ugen0.2: <vendor 0x8087> at usbus0
uhub2: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on usbus1
Root mount waiting for: usbus1 usbus0
uhub2: 6 ports with 6 removable, self powered
uhub3: 8 ports with 8 removable, self powered
ugen1.3: <ServerEngines> at usbus1
ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on usbus1
kbd2 at ukbd0
Trying to mount root from ufs:/dev/mirror/gm2a

# pciconf -lv
hostb0@pci0:0:0:0:      class=0x060000 card=0x3318103c chip=0xd1308086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:3:0:       class=0x060400 card=0x3318103c chip=0xd1388086 rev=0x11 hdr=0x01
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-PCI
none0@pci0:0:8:0:       class=0x088000 card=0x00000000 chip=0xd1558086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = base peripheral
none1@pci0:0:8:1:       class=0x088000 card=0x00000000 chip=0xd1568086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = base peripheral
none2@pci0:0:8:2:       class=0x088000 card=0x00000000 chip=0xd1578086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = base peripheral
none3@pci0:0:8:3:       class=0x088000 card=0x00000000 chip=0xd1588086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = base peripheral
none4@pci0:0:16:0:      class=0x088000 card=0x00000000 chip=0xd1508086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = base peripheral
none5@pci0:0:16:1:      class=0x088000 card=0x00000000 chip=0xd1518086 rev=0x11 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = base peripheral
ehci0@pci0:0:26:0:      class=0x0c0320 card=0x3118103c chip=0x3b3c8086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = serial bus
    subclass   = USB
pcib2@pci0:0:28:0:      class=0x060400 card=0x3318103c chip=0x3b428086 rev=0x05 hdr=0x01
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-PCI
pcib3@pci0:0:28:4:      class=0x060400 card=0x3318103c chip=0x3b4a8086 rev=0x05 hdr=0x01
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:28:5:      class=0x060400 card=0x3318103c chip=0x3b4c8086 rev=0x05 hdr=0x01
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-PCI
pcib5@pci0:0:28:6:      class=0x060400 card=0x3318103c chip=0x3b4e8086 rev=0x05 hdr=0x01
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-PCI
pcib6@pci0:0:28:7:      class=0x060400 card=0x3318103c chip=0x3b508086 rev=0x05 hdr=0x01
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-PCI
ehci1@pci0:0:29:0:      class=0x0c0320 card=0x3118103c chip=0x3b348086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = serial bus
    subclass   = USB
pcib7@pci0:0:30:0:      class=0x060401 card=0x3318103c chip=0x244e8086 rev=0xa5 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
isab0@pci0:0:31:0:      class=0x060100 card=0x3118103c chip=0x3b148086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = bridge
    subclass   = PCI-ISA
atapci0@pci0:0:31:2:    class=0x010601 card=0x3118103c chip=0x3b228086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'IBEX AHCI Controller(6Port)'
    class      = mass storage
    subclass   = SATA
ichsmb0@pci0:0:31:3:    class=0x0c0500 card=0x3318103c chip=0x3b308086 rev=0x05 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = serial bus
    subclass   = SMBus
bge0@pci0:32:0:0:       class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10 hdr=0x00
    vendor     = 'Broadcom Corporation'
    class      = network
    subclass   = ethernet
bge1@pci0:34:0:0:       class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10 hdr=0x00
    vendor     = 'Broadcom Corporation'
    class      = network
    subclass   = ethernet
vgapci0@pci0:36:0:0:    class=0x030000 card=0x31fa103c chip=0x0522102b rev=0x02 hdr=0x00
    vendor     = 'Matrox Electronic Systems Ltd.'
    device     = 'Matrox G200e (ServerEngines) - English (G200e)'
    class      = display
    subclass   = VGA

Server model: ProLiant DL120 G6
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: brucec 
Responsible-Changed-When: Tue Nov 16 13:46:07 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152295 
State-Changed-From-To: open->feedback 
State-Changed-By: yongari 
State-Changed-When: Fri Nov 19 17:53:56 UTC 2010 
State-Changed-Why:  
Could you try latest stable/8 on the box? There were numerous 
stability fixes as well as new features. 
Also show me the output of "vmstat -i". 


Responsible-Changed-From-To: freebsd-net->yongari 
Responsible-Changed-By: yongari 
Responsible-Changed-When: Fri Nov 19 17:53:56 UTC 2010 
Responsible-Changed-Why:  
Grab. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152295 

From: Mykola Zubach <zuborg@gmail.com>
To: bug-followup@FreeBSD.org, zuborg@gmail.com
Cc:  
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 on high bandwidth for BCM5784, 7.3-R works
Date: Wed, 24 Nov 2010 15:35:48 +0200

 --000e0cd519e42fa6da0495cc9124
 Content-Type: text/plain; charset=ISO-8859-1
 
 >Could you try latest stable/8 on the box? There were numerous
 stability fixes as well as new features.
 >Also show me the output of "vmstat -i".
 
 8.1-STABLE at 2010-11-22 works stable, no problems with networking.
 PR should be closed.
 
 Just disabled TSO feature on bge0 in order to reduce cpu usage by '{swi1:
 netisr 0}' kernel thread,
 which was about 80% ob 600Mbit/s. POLLING is used due more efficient cpu
 usage.
 
 # vmstat -i
 interrupt total rate
 irq16: bge0 ehci0 14426 1
 irq23: ehci1 18671 1
 cpu0: timer 19014240 2020
 irq256: ahci0 6844301 727
 cpu3: timer 19012976 2020
 cpu2: timer 19012978 2020
 cpu1: timer 19013000 2020
 Total 82930592 8811
 
 --000e0cd519e42fa6da0495cc9124
 Content-Type: text/html; charset=ISO-8859-1
 Content-Transfer-Encoding: quoted-printable
 
 <div>&gt;Could you try latest stable/8 on the box? There were numerous=A0<b=
 r></div>stability fixes as well as new features. <br><div>&gt;Also show me =
 the output of &quot;vmstat -i&quot;.</div><div><br></div><div>8.1-STABLE at=
  2010-11-22 works stable, no problems with networking.</div>
 <div>PR should be closed.</div><div><br></div><div>Just disabled TSO featur=
 e on bge0 in order to reduce cpu usage by &#39;{swi1: netisr 0}&#39; kernel=
  thread,</div><div>which was about 80% ob 600Mbit/s. POLLING is used due mo=
 re efficient cpu usage.</div>
 <div><br></div><div># vmstat -i<br>interrupt                          total=
        rate<br>irq16: bge0 ehci0                  14426          1<br>irq23=
 : ehci1                       18671          1<br>cpu0: timer              =
        19014240       2020<br>
 irq256: ahci0                    6844301        727<br>cpu3: timer         =
             19012976       2020<br>cpu2: timer                     19012978=
        2020<br>cpu1: timer                     19013000       2020<br>Total=
                            82930592       8811<br>
 <br></div>
 
 --000e0cd519e42fa6da0495cc9124--

From: Pyun YongHyeon <pyunyh@gmail.com>
To: Mykola Zubach <zuborg@gmail.com>
Cc: yongari@freebsd.org, bug-followup@FreeBSD.org
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works
Date: Wed, 24 Nov 2010 10:01:51 -0800

 On Wed, Nov 24, 2010 at 02:10:12PM +0000, Mykola Zubach wrote:
 
 [...]
 
 >  8.1-STABLE at 2010-11-22 works stable, no problems with networking.
 >  PR should be closed.
 >  
 
 Thanks for testing!
 
 >  Just disabled TSO feature on bge0 in order to reduce cpu usage by '{swi1:
 >  netisr 0}' kernel thread,
 >  which was about 80% ob 600Mbit/s. POLLING is used due more efficient cpu
 >  usage.
 >  
 
 It's hard to tell what is happening here without knowing details
 but 80% for 600Mbps looks not normal. Are you seeing 80% of CPU
 time is consumed in interrupts? Using POLLING on intelligent
 controllers like bge(4) is not recommended. There could be some
 situations where POLLING would give better result but most cases
 polling just adds additional delays and system load.
 
 >  # vmstat -i
 >  interrupt total rate
 >  irq16: bge0 ehci0 14426 1
 
 Sharing interrupts with other slow devices is not good thing. I
 thought BCM5784 has MSI capability. Did you disable MSI?
 Would you show me the output of "pciconf -lcbv" and dmesg output on
 stable/8?
 
 >  irq23: ehci1 18671 1
 >  cpu0: timer 19014240 2020
 >  irq256: ahci0 6844301 727
 >  cpu3: timer 19012976 2020
 >  cpu2: timer 19012978 2020
 >  cpu1: timer 19013000 2020
 >  Total 82930592 8811
 >  

From: "Laust S. Jespersen" <l@ust.dk>
To: bug-followup@FreeBSD.org, zuborg@gmail.com
Cc:  
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 on high bandwidth for BCM5784, 7.3-R works
Date: Wed, 24 Nov 2010 21:38:57 +0100

 We are seing the exact same issue on our IBM Blade servers, the
 upgrade to 8.2-PRE
 did however seem to do the trick on our dell R300 servers
 (which does not have the 5704 chipset)
 
 first the IBM server:
 interrupt                          total       rate
 irq77: bge0                     27595847       2669
 irq24: mpt0                        41895          4
 irq16: uhci0                        7888          0
 cpu0: timer                     20669838       1999
 cpu1: timer                     20655221       1997
 cpu3: timer                     20655339       1998
 cpu2: timer                     20654616       1997
 Total                          110280644      10667
 ---
 
 Copyright (c) 1992-2010 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
 FreeBSD is a registered trademark of The FreeBSD Foundation.
 FreeBSD 8.2-PRERELEASE #0: Tue Nov 23 03:04:19 CET 2010
     root@dns1-albx.telenor.dk:/usr/obj/usr/src/sys/GENERIC amd64
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.12-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0xf4a  Family = f  Model = 4  Stepping = 10
   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
   Features2=0x649d<SSE3,DTES64,MON,DS_CPL,EST,CNXT-ID,CX16,xTPR>
   AMD Features=0x20000800<SYSCALL,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant
 real memory  = 1073741824 (1024 MB)
 avail memory = 1017008128 (969 MB)
 ACPI APIC Table: <IBM    SERBLADE>
 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 FreeBSD/SMP: 2 package(s) x 1 core(s) x 2 HTT threads
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP/HT): APIC ID:  1
  cpu2 (AP): APIC ID:  6
  cpu3 (AP/HT): APIC ID:  7
 ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default
 16 (20101013/tbfadt-707)
 ioapic3 <Version 2.0> irqs 72-95 on motherboard
 ioapic2 <Version 2.0> irqs 48-71 on motherboard
 ioapic1 <Version 2.0> irqs 24-47 on motherboard
 ioapic0 <Version 2.0> irqs 0-23 on motherboard
 kbd1 at kbdmux0
 acpi0: <IBM SERBLADE> on motherboard
 acpi0: [ITHREAD]
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x588-0x58b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 cpu2: <ACPI CPU> on acpi0
 cpu3: <ACPI CPU> on acpi0
 pcib0: <ACPI Host-PCI bridge> on acpi0
 pci0: <ACPI PCI bus> on pcib0
 pci0: <unknown> at device 0.1 (no driver attached)
 pcib1: <ACPI PCI-PCI bridge> at device 3.0 on pci0
 pci4: <ACPI PCI bus> on pcib1
 pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci4
 pci6: <ACPI PCI bus> on pcib2
 pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci4
 pci5: <ACPI PCI bus> on pcib3
 bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
 0x002100> mem 0xdcff0000-0xdcffffff irq 77 at device 1.0 on pci5
 bge0: CHIP ID 0x00002100; ASIC REV 0x02; CHIP REV 0x21; PCI-X
 bge0: Ethernet address: 00:14:5e:be:16:c0
 bge0: [ITHREAD]
 bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
 0x002100> mem 0xdcfe0000-0xdcfeffff irq 78 at device 1.1 on pci5
 bge1: CHIP ID 0x00002100; ASIC REV 0x02; CHIP REV 0x21; PCI-X
 bge1: Ethernet address: 00:14:5e:be:16:c1
 bge1: [ITHREAD]
 pci0: <base peripheral> at device 8.0 (no driver attached)
 pcib4: <ACPI PCI-PCI bridge> at device 28.0 on pci0
 pci2: <ACPI PCI bus> on pcib4
 mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x4000-0x40ff mem
 0xdeff0000-0xdeffffff,0xdefe0000-0xdefeffff irq 24 at device 1.0 on
 pci2
 mpt0: [ITHREAD]
 mpt0: MPI Version=1.2.15.0
 mpt0: Capabilities: ( RAID-1E RAID-1 SAFTE )
 mpt0: 1 Active Volume (1 Max)
 mpt0: 2 Hidden Drive Members (6 Max)
 uhci0: <UHCI (generic) USB controller> port 0x2200-0x221f irq 16 at
 device 29.0 on pci0
 uhci0: [ITHREAD]
 usbus0: <UHCI (generic) USB controller> on uhci0
 uhci1: <UHCI (generic) USB controller> port 0x2600-0x261f irq 19 at
 device 29.1 on pci0
 uhci1: [ITHREAD]
 usbus1: <UHCI (generic) USB controller> on uhci1
 pci0: <base peripheral> at device 29.4 (no driver attached)
 pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci1: <ACPI PCI bus> on pcib5
 vgapci0: <VGA-compatible display> port 0x3000-0x30ff mem
 0xf0000000-0xf7ffffff,0xf8000000-0xf800ffff irq 20 at device 1.0 on
 pci1
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel 6300ESB UDMA100 controller> port
 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376 at device 31.1 on pci0
 ata0: <ATA channel 0> on atapci0
 ata0: [ITHREAD]
 ata1: <ATA channel 1> on atapci0
 ata1: [ITHREAD]
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff
 irq 4 flags 0x10 on acpi0
 uart0: [FILTER]
 uart0: console (9600,n,8,1)
 atrtc0: <AT realtime clock> port 0x70-0x73 irq 8 on acpi0
 orm0: <ISA Option ROM> at iomem 0xc0000-0xc8fff on isa0
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x300>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 atkbd0: [ITHREAD]
 psm0: unable to set the command byte.
 ppc0: cannot reserve I/O port range
 est0: <Enhanced SpeedStep Frequency Control> on cpu0
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr f1c00000f1c
 device_attach: est0 attach returned 6
 p4tcc0: <CPU Frequency Thermal Control> on cpu0
 est1: <Enhanced SpeedStep Frequency Control> on cpu1
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr f1c00000f1c
 device_attach: est1 attach returned 6
 p4tcc1: <CPU Frequency Thermal Control> on cpu1
 est2: <Enhanced SpeedStep Frequency Control> on cpu2
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr f1c00000f1c
 device_attach: est2 attach returned 6
 p4tcc2: <CPU Frequency Thermal Control> on cpu2
 est3: <Enhanced SpeedStep Frequency Control> on cpu3
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr f1c00000f1c
 device_attach: est3 attach returned 6
 p4tcc3: <CPU Frequency Thermal Control> on cpu3
 Timecounters tick every 1.000 msec
 usbus0: 12Mbps Full Speed USB v1.0
 usbus1: 12Mbps Full Speed USB v1.0
 ugen0.1: <Intel> at usbus0
 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
 ugen1.1: <Intel> at usbus1
 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
 uhub0: 2 ports with 2 removable, self powered
 uhub1: 2 ports with 2 removable, self powered
 mpt0:vol0(mpt0:0:0): Settings ( Hot-Plug-Spares )
 mpt0:vol0(mpt0:0:0): Using Spare Pool: 0
 mpt0:vol0(mpt0:0:0): 2 Members:
       (mpt0:1:0:0): Primary Online
       (mpt0:1:0:0): Secondary Online
 mpt0:vol0(mpt0:0:0): RAID-1 - Degraded
 mpt0:vol0(mpt0:0:0): Status ( Enabled )
 (mpt0:vol0:0): Physical (mpt0:0:0:0), Pass-thru (mpt0:1:0:0)
 (mpt0:vol0:0): Online
 ugen0.2: <vendor 0x04b4> at usbus0
 uhub2: <vendor 0x04b4 product 0x6560, class 9/0, rev 2.00/0.07, addr
 2> on usbus0
 uhub2: 3 ports with 3 removable, self powered
 ugen0.3: <IBM Corpation-xSeries> at usbus0
 umass0: <IBM Corpation-xSeries USB2.0 Storage Device, class 0/0, rev
 2.00/0.01, addr 3> on usbus0
 umass0:  SCSI over Bulk-Only; quirks = 0x4100
 umass0:2:0:-1: Attached to scbus2
 (probe2:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0
 (probe2:umass-sim0:0:0:0): CAM status: SCSI Status Error
 (probe2:umass-sim0:0:0:0): SCSI status: Check Condition
 (probe2:umass-sim0:0:0:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power
 on, reset, or bus device reset occurred)
 (probe2:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0
 (probe2:umass-sim0:0:0:0): CAM status: SCSI Status Error
 (probe2:umass-sim0:0:0:0): SCSI status: Check Condition
 (probe2:umass-sim0:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not
 present - tray closed)
 ugen0.4: <IBM> at usbus0
 ukbd0: <HID KB> on usbus0
 kbd2 at ukbd0
 ums0: <HID MS> on usbus0
 ums0: 3 buttons and [XYZ] coordinates ID=0
 pass1 at mpt0 bus 1 scbus1 target 0 lun 0
 pass1: <IBM-ESXS ST973401LC    FN B41D> Fixed Uninstalled SCSI-4 device
 pass1: 320.000MB/s transfers (160.000MHz, offset 63, 16bit)
 pass1: Command Queueing enabled
 da0 at mpt0 bus 0 scbus0 target 0 lun 0
 da0: <LSILOGIC 1030 IM       IM 1000> Fixed Direct Access SCSI-2 device
 da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit)
 da0: Command Queueing enabled
 da0: 69878MB (143110144 512 byte sectors: 255H 63S/T 8908C)
 SMP: AP CPU #1 Launched!cd0 at umass-sim0 bus 0 scbus2 target 0 lun 0
 SMP: AP CPU #3 Launched!
 cd0:
 SMP: AP CPU #2 Launched!<MATSHITA UJDA770 DVD/CDRW 1.20> Removable
 CD-ROM SCSI-0 device
 
 cd0: 1.000MB/s transfers
 cd0: Attempt to query device size failed: NOT READY, Medium not
 present - tray closed
 Trying to mount root from ufs:/dev/da0s1a
 WARNING: / was not properly dismounted
 WARNING: /home was not properly dismounted
 WARNING: /tmp was not properly dismounted
 WARNING: /usr was not properly dismounted
 WARNING: /var was not properly dismounted
 ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based
 forwarding disabled, default to deny, logging disabled
 
 ---
 hostb0@pci0:0:0:0:      class=0x060000 card=0x02dd1014 chip=0x35908086
 rev=0x0c hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'E7520 Server Memory Controller Hub'
     class      = bridge
     subclass   = HOST-PCI
 none0@pci0:0:0:1:       class=0xff0000 card=0x02dd1014 chip=0x35918086
 rev=0x0c hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'E752x Memory Controller Hub Error Reporting Registers'
 pcib1@pci0:0:3:0:       class=0x060400 card=0x00000000 chip=0x35968086
 rev=0x0c hdr=0x01
     vendor     = 'Intel Corporation'
     device     = 'E752x Memory Controller Hub PCIe Port A1'
     class      = bridge
     subclass   = PCI-PCI
 none1@pci0:0:8:0:       class=0x088000 card=0x02dd1014 chip=0x359b8086
 rev=0x0c hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'E752x Memory Controller Hub Extended Configuration Registers'
     class      = base peripheral
 pcib4@pci0:0:28:0:      class=0x060400 card=0x00000000 chip=0x25ae8086
 rev=0x02 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = 'Hub Interface to PCI-X Bridge (6300ESB)'
     class      = bridge
     subclass   = PCI-PCI
 uhci0@pci0:0:29:0:      class=0x0c0300 card=0x02dd1014 chip=0x25a98086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'USB 1.1 UHCI Controller *1 (6300ESB)'
     class      = serial bus
     subclass   = USB
 uhci1@pci0:0:29:1:      class=0x0c0300 card=0x02dd1014 chip=0x25aa8086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'USB 1.1 UHCI Controller *2 (6300ESB)'
     class      = serial bus
     subclass   = USB
 none2@pci0:0:29:4:      class=0x088000 card=0x02dd1014 chip=0x25ab8086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'Watchdog Timer (6300ESB)'
     class      = base peripheral
 ioapic0@pci0:0:29:5:    class=0x080020 card=0x02dd1014 chip=0x25ac8086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '6300ESB I/O Advanced Programmable Interrupt Controller'
     class      = base peripheral
     subclass   = interrupt controller
 pcib5@pci0:0:30:0:      class=0x060400 card=0x00000000 chip=0x244e8086
 rev=0x0a hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub
 Interface to PCI Bridge'
     class      = bridge
     subclass   = PCI-PCI
 isab0@pci0:0:31:0:      class=0x060100 card=0x00000000 chip=0x25a18086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '6300ESB LPC Inteface Controller'
     class      = bridge
     subclass   = PCI-ISA
 atapci0@pci0:0:31:1:    class=0x01018a card=0x02dd1014 chip=0x25a28086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'IDE Controller (6300ESB)'
     class      = mass storage
     subclass   = ATA
 none3@pci0:0:31:3:      class=0x0c0500 card=0x02dd1014 chip=0x25a48086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = 'SMBus Controller (6300ESB)'
     class      = serial bus
     subclass   = SMBus
 pcib2@pci0:4:0:0:       class=0x060400 card=0x00000000 chip=0x03298086
 rev=0x09 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = 'PCI Express-to-PCI Express Bridge A (6700PXH)'
     class      = bridge
     subclass   = PCI-PCI
 pcib3@pci0:4:0:2:       class=0x060400 card=0x00000000 chip=0x032a8086
 rev=0x09 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = 'PCI Express-to-PCI Express Bridge B (6700PXH)'
     class      = bridge
     subclass   = PCI-PCI
 bge0@pci0:5:1:0:        class=0x020000 card=0x02e81014 chip=0x16a814e4
 rev=0x10 hdr=0x00
     vendor     = 'Broadcom Corporation'
     device     = 'NetXtreme Gigabit Ethernet (BCM5704)'
     class      = network
     subclass   = ethernet
 bge1@pci0:5:1:1:        class=0x020000 card=0x02e81014 chip=0x16a814e4
 rev=0x10 hdr=0x00
     vendor     = 'Broadcom Corporation'
     device     = 'NetXtreme Gigabit Ethernet (BCM5704)'
     class      = network
     subclass   = ethernet
 mpt0@pci0:2:1:0:        class=0x010000 card=0x026d1014 chip=0x00301000
 rev=0x08 hdr=0x00
     vendor     = 'LSI Logic (Was: Symbios Logic, NCR)'
     device     = 'PCI-X to Ultra320 SCSI Controller (LSI53C1020/1030)'
     class      = mass storage
     subclass   = SCSI
 vgapci0@pci0:1:1:0:     class=0x030000 card=0x02c81014 chip=0x51591002
 rev=0x00 hdr=0x00
     vendor     = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.'
     device     = 'Radeon 7000 / Radeon VE (RV100)'
     class      = display
     subclass   = VGA
 
 Now the Dell:
 interrupt                          total       rate
 irq16: mpt0                       116393          0
 irq20: uhci1                        1158          0
 irq21: uhci0 uhci2+                70708          0
 irq23: atapci0                        35          0
 cpu0: timer                    282754852       1999
 irq256: bge0                    89773965        634
 cpu3: timer                    282535623       1998
 cpu1: timer                    282746350       1999
 cpu2: timer                    282534889       1998
 Total                         1220533973       8632
 ---
 Copyright (c) 1992-2010 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
 FreeBSD is a registered trademark of The FreeBSD Foundation.
 FreeBSD 8.2-PRERELEASE #1: Tue Nov 23 05:51:42 CET 2010
     root@dns1-oers.telenor.dk:/usr/obj/usr/src/sys/GENERIC amd64
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Xeon(R) CPU           X3323  @ 2.50GHz (2500.01-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x10676  Family = 6  Model = 17  Stepping = 6
   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
   Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
   AMD Features=0x20100800<SYSCALL,NX,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant
 real memory  = 4294967296 (4096 MB)
 avail memory = 4095877120 (3906 MB)
 ACPI APIC Table: <DELL   PE_SC3  >
 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 FreeBSD/SMP: 1 package(s) x 4 core(s)
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
  cpu2 (AP): APIC ID:  2
  cpu3 (AP): APIC ID:  3
 ioapic0: Changing APIC ID to 4
 ioapic0 <Version 2.0> irqs 0-23 on motherboard
 kbd1 at kbdmux0
 acpi0: <DELL PE_SC3> on motherboard
 acpi0: [ITHREAD]
 acpi0: Power Button (fixed)
 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 cpu2: <ACPI CPU> on acpi0
 cpu3: <ACPI CPU> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 pcib1: <PCI-PCI bridge> at device 2.0 on pci0
 pci3: <PCI bus> on pcib1
 pcib2: <PCI-PCI bridge> at device 3.0 on pci0
 pci4: <PCI bus> on pcib2
 pcib3: <ACPI PCI-PCI bridge> at device 4.0 on pci0
 pci5: <ACPI PCI bus> on pcib3
 mpt0: <LSILogic SAS/SATA Adapter> port 0xec00-0xecff mem
 0xdfcec000-0xdfceffff,0xdfcf0000-0xdfcfffff irq 16 at device 0.0 on
 pci5
 mpt0: [ITHREAD]
 mpt0: MPI Version=1.5.14.0
 mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
 mpt0: 1 Active Volume (2 Max)
 mpt0: 2 Hidden Drive Members (14 Max)
 pcib4: <PCI-PCI bridge> at device 5.0 on pci0
 pci6: <PCI bus> on pcib4
 pcib5: <ACPI PCI-PCI bridge> at device 6.0 on pci0
 pci7: <ACPI PCI bus> on pcib5
 pcib6: <ACPI PCI-PCI bridge> at device 7.0 on pci0
 pci8: <ACPI PCI bus> on pcib6
 pcib7: <PCI-PCI bridge> irq 16 at device 28.0 on pci0
 pci9: <PCI bus> on pcib7
 pcib8: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
 pci1: <ACPI PCI bus> on pcib8
 bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
 0x00a200> mem 0xdfdf0000-0xdfdfffff irq 16 at device 0.0 on pci1
 bge0: CHIP ID 0x0000a200; ASIC REV 0x0a; CHIP REV 0xa2; PCI-E
 miibus0: <MII bus> on bge0
 brgphy0: <BCM5722 10/100/1000baseTX PHY> PHY 1 on miibus0
 brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
 1000baseT-FDX, auto
 bge0: Ethernet address: 00:1e:c9:cd:da:92
 bge0: [FILTER]
 pcib9: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
 pci2: <ACPI PCI bus> on pcib9
 bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
 0x00a200> mem 0xdfef0000-0xdfefffff irq 17 at device 0.0 on pci2
 bge1: CHIP ID 0x0000a200; ASIC REV 0x0a; CHIP REV 0xa2; PCI-E
 miibus1: <MII bus> on bge1
 brgphy1: <BCM5722 10/100/1000baseTX PHY> PHY 1 on miibus1
 brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
 1000baseT-FDX, auto
 bge1: Ethernet address: 00:1e:c9:cd:da:93
 bge1: [FILTER]
 uhci0: <Intel 82801I (ICH9) USB controller> port 0xcc80-0xcc9f irq 21
 at device 29.0 on pci0
 uhci0: [ITHREAD]
 usbus0: <Intel 82801I (ICH9) USB controller> on uhci0
 uhci1: <Intel 82801I (ICH9) USB controller> port 0xcca0-0xccbf irq 20
 at device 29.1 on pci0
 uhci1: [ITHREAD]
 usbus1: <Intel 82801I (ICH9) USB controller> on uhci1
 uhci2: <Intel 82801I (ICH9) USB controller> port 0xccc0-0xccdf irq 21
 at device 29.2 on pci0
 uhci2: [ITHREAD]
 usbus2: <Intel 82801I (ICH9) USB controller> on uhci2
 ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem
 0xdfaffc00-0xdfafffff irq 21 at device 29.7 on pci0
 ehci0: [ITHREAD]
 usbus3: EHCI version 1.0
 usbus3: <Intel 82801I (ICH9) USB 2.0 controller> on ehci0
 pcib10: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci10: <ACPI PCI bus> on pcib10
 vgapci0: <VGA-compatible display> port 0xdc00-0xdcff mem
 0xd0000000-0xd7ffffff,0xdfff0000-0xdfffffff irq 19 at device 7.0 on
 pci10
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH9 SATA300 controller> port
 0xcc20-0xcc27,0xcc10-0xcc13,0xcc28-0xcc2f,0xcc14-0xcc17,0xcc40-0xcc4f,0xcc50-0xcc5f
 irq 23 at device 31.2 on pci0
 atapci0: [ITHREAD]
 ata2: <ATA channel 0> on atapci0
 ata2: [ITHREAD]
 ata3: <ATA channel 1> on atapci0
 ata3: [ITHREAD]
 atapci1: <Intel ICH9 SATA300 controller> port
 0xcc30-0xcc37,0xcc18-0xcc1b,0xcc38-0xcc3f,0xcc1c-0xcc1f,0xcc60-0xcc6f,0xcc70-0xcc7f
 irq 22 at device 31.5 on pci0
 atapci1: [ITHREAD]
 ata4: <ATA channel 0> on atapci1
 ata4: [ITHREAD]
 ata5: <ATA channel 1> on atapci1
 ata5: [ITHREAD]
 acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
 Timecounter "HPET" frequency 14318180 Hz quality 900
 atrtc0: <AT realtime clock> port 0x70-0x7f irq 8 on acpi0
 fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
 fdc0: does not respond
 device_attach: fdc0 attach returned 6
 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 uart0: [FILTER]
 uart0: console (9600,n,8,1)
 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
 uart1: [FILTER]
 orm0: <ISA Option ROMs> at iomem
 0xc0000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcbfff,0xec000-0xeffff on
 isa0
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x300>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 atkbd0: [ITHREAD]
 ppc0: cannot reserve I/O port range
 est0: <Enhanced SpeedStep Frequency Control> on cpu0
 p4tcc0: <CPU Frequency Thermal Control> on cpu0
 est1: <Enhanced SpeedStep Frequency Control> on cpu1
 p4tcc1: <CPU Frequency Thermal Control> on cpu1
 est2: <Enhanced SpeedStep Frequency Control> on cpu2
 p4tcc2: <CPU Frequency Thermal Control> on cpu2
 est3: <Enhanced SpeedStep Frequency Control> on cpu3
 p4tcc3: <CPU Frequency Thermal Control> on cpu3
 Timecounters tick every 1.000 msec
 usbus0: 12Mbps Full Speed USB v1.0
 usbus1: 12Mbps Full Speed USB v1.0
 usbus2: 12Mbps Full Speed USB v1.0
 usbus3: 480Mbps High Speed USB v2.0
 mpt0:vol0(mpt0:0:0): ugen0.1: <Intel> at usbus0
 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
 ugen1.1: <Intel> at usbus1
 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
 ugen2.1: <Intel> at usbus2
 uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
 ugen3.1: <Intel> at usbus3
 uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
 Settings ( Member-WCE Hot-Plug-Spares High-Priority-ReSync )
 mpt0:vol0(mpt0:0:0): Using Spare Pool: 0
 mpt0:vol0(mpt0:0:0): 2 Members:
       (mpt0:1:9:0): Primary Online
       (mpt0:1:1:0): Secondary Online
 mpt0:vol0(mpt0:0:0): RAID-1 - Optimal
 mpt0:vol0(mpt0:0:0): Status ( Enabled )
 (mpt0:vol0:1): Physical (mpt0:0:1:0), Pass-thru (mpt0:1:0:0)
 (mpt0:vol0:1): Online
 (mpt0:vol0:0): Physical (mpt0:0:9:0), Pass-thru (mpt0:1:1:0)
 (mpt0:vol0:0): Online
 acd0: DVDROM <TEAC DVD-ROM DV28SV/D.0E> at ata2-slave UDMA100 SATA 1.5Gb/s
 uhub0: 2 ports with 2 removable, self powered
 uhub1: 2 ports with 2 removable, self powered
 uhub2: 2 ports with 2 removable, self powered
 uhub3: 6 ports with 6 removable, self powered
 ugen3.2: <vendor 0x04b4> at usbus3
 uhub4: <vendor 0x04b4 product 0x6560, class 9/0, rev 2.00/90.15, addr
 2> on usbus3
 ses0 at mpt0 bus 0 scbus0 target 8 lun 0
 ses0: <DP BACKPLANE 1.05> Fixed Enclosure Services SCSI-5 device
 ses0: 300.000MB/s transfers
 ses0: SCSI-3 SES Device
 pass2 at mpt0 bus 1 scbus1 target 0 lun 0
 pass2: <ATA ST3250620NS 3BKS> Fixed Uninstalled SCSI-5 device
 pass2: 300.000MB/s transfers
 pass2: Command Queueing enabled
 da0 at mpt0 bus 0 scbus0 target 0 lun 0
 da0: <Dell VIRTUAL DISK 1028> Fixed Direct Access SCSI-5 device
 da0: 300.000MB/s transfersugen1.2: <ST Microelectronics> at usbus1
 
 da0: Command Queueing enabled
 da0: 237464MB (486326272 512 byte sectors: 255H 63S/T 30272C)
 SMP: AP CPU #3 Launched!
 SMP: AP CPU #1 Launched!
 SMP: AP CPU #2 Launched!
 ukbd0: <USB Keyboard> on usbus1
 kbd2 at ukbd0
 ums0: <ST Microelectronics P2-CIM-USB, class 0/0, rev 1.00/0.00, addr
 2> on usbus1
 ums0: error reading report description
 device_attach: ums0 attach returned 12
 ums0: <ST Microelectronics P2-CIM-USB, class 0/0, rev 1.00/0.00, addr
 2> on usbus1
 ums0: error reading report description
 device_attach: ums0 attach returned 12
 uhub4: 2 ports with 2 removable, self powered
 Trying to mount root from ufs:/dev/da0s1a
 ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based
 forwarding disabled, default to deny, logging disabled
 Limiting icmp unreach response from 220 to 200 packets/sec
 ----
 hostb0@pci0:0:0:0:      class=0x060000 card=0x80868086 chip=0x65c08086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset Memory Controller Hub (??)'
     class      = bridge
     subclass   = HOST-PCI
 pcib1@pci0:0:2:0:       class=0x060400 card=0x00000000 chip=0x65f78086
 rev=0x90 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset PCIe x8 Port 2-3'
     class      = bridge
     subclass   = PCI-PCI
 pcib2@pci0:0:3:0:       class=0x060400 card=0x00000000 chip=0x65e38086
 rev=0x90 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset PCIe x4 Port 3'
     class      = bridge
     subclass   = PCI-PCI
 pcib3@pci0:0:4:0:       class=0x060400 card=0x00000000 chip=0x65f88086
 rev=0x90 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset PCIe x8 Port 4-5'
     class      = bridge
     subclass   = PCI-PCI
 pcib4@pci0:0:5:0:       class=0x060400 card=0x00000000 chip=0x65e58086
 rev=0x90 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset PCIe x4 Port 5'
     class      = bridge
     subclass   = PCI-PCI
 pcib5@pci0:0:6:0:       class=0x060400 card=0x00000000 chip=0x65f98086
 rev=0x90 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset PCIe x8 Port 6-7'
     class      = bridge
     subclass   = PCI-PCI
 pcib6@pci0:0:7:0:       class=0x060400 card=0x00000000 chip=0x65e78086
 rev=0x90 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset PCIe x4 Port 7'
     class      = bridge
     subclass   = PCI-PCI
 hostb1@pci0:0:16:0:     class=0x060000 card=0x020f1028 chip=0x65f08086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset FSB Registers'
     class      = bridge
     subclass   = HOST-PCI
 hostb2@pci0:0:16:1:     class=0x060000 card=0x020f1028 chip=0x65f08086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset FSB Registers'
     class      = bridge
     subclass   = HOST-PCI
 hostb3@pci0:0:16:2:     class=0x060000 card=0x020f1028 chip=0x65f08086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset FSB Registers'
     class      = bridge
     subclass   = HOST-PCI
 hostb4@pci0:0:17:0:     class=0x060000 card=0x80868086 chip=0x65f18086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset Reserved Registers'
     class      = bridge
     subclass   = HOST-PCI
 hostb5@pci0:0:19:0:     class=0x060000 card=0x80868086 chip=0x65f38086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset Miscellaneous Registers'
     class      = bridge
     subclass   = HOST-PCI
 hostb6@pci0:0:21:0:     class=0x060000 card=0x80868086 chip=0x65f58086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset DDR Channel 0 Registers'
     class      = bridge
     subclass   = HOST-PCI
 hostb7@pci0:0:22:0:     class=0x060000 card=0x80868086 chip=0x65f68086
 rev=0x90 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '5100 Chipset DDR Channel 1 Registers'
     class      = bridge
     subclass   = HOST-PCI
 pcib7@pci0:0:28:0:      class=0x060400 card=0x020f1028 chip=0x29408086
 rev=0x02 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1'
     class      = bridge
     subclass   = PCI-PCI
 pcib8@pci0:0:28:4:      class=0x060400 card=0x020f1028 chip=0x29488086
 rev=0x02 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 5'
     class      = bridge
     subclass   = PCI-PCI
 pcib9@pci0:0:28:5:      class=0x060400 card=0x020f1028 chip=0x294a8086
 rev=0x02 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) PCIe Root Port 6'
     class      = bridge
     subclass   = PCI-PCI
 uhci0@pci0:0:29:0:      class=0x0c0300 card=0x020f1028 chip=0x29348086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
     class      = serial bus
     subclass   = USB
 uhci1@pci0:0:29:1:      class=0x0c0300 card=0x020f1028 chip=0x29358086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
     class      = serial bus
     subclass   = USB
 uhci2@pci0:0:29:2:      class=0x0c0300 card=0x020f1028 chip=0x29368086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) USB Universal Host Controller'
     class      = serial bus
     subclass   = USB
 ehci0@pci0:0:29:7:      class=0x0c0320 card=0x020f1028 chip=0x293a8086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Controller'
     class      = serial bus
     subclass   = USB
 pcib10@pci0:0:30:0:     class=0x060401 card=0x00000000 chip=0x244e8086
 rev=0x92 hdr=0x01
     vendor     = 'Intel Corporation'
     device     = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub
 Interface to PCI Bridge'
     class      = bridge
     subclass   = PCI-PCI
 isab0@pci0:0:31:0:      class=0x060100 card=0x00000000 chip=0x29168086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IR (ICH9R) LPC Interface Controller'
     class      = bridge
     subclass   = PCI-ISA
 atapci0@pci0:0:31:2:    class=0x01018f card=0x020f1028 chip=0x29208086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) 4 port Serial ATA
 Storage Controller 1'
     class      = mass storage
     subclass   = ATA
 atapci1@pci0:0:31:5:    class=0x010185 card=0x020f1028 chip=0x29268086
 rev=0x02 hdr=0x00
     vendor     = 'Intel Corporation'
     device     = '82801IB/IR/IH (ICH9 Family) 2 port Serial ATA
 Storage Controller 2'
     class      = mass storage
     subclass   = ATA
 mpt0@pci0:5:0:0:        class=0x010000 card=0x1f0e1028 chip=0x00581000
 rev=0x08 hdr=0x00
     vendor     = 'LSI Logic (Was: Symbios Logic, NCR)'
     device     = 'SAS 3000 series, 8-port with 1068E -StorPort'
     class      = mass storage
     subclass   = SCSI
 bge0@pci0:1:0:0:        class=0x020000 card=0x020f1028 chip=0x165a14e4
 rev=0x00 hdr=0x00
     vendor     = 'Broadcom Corporation'
     device     = 'Broadcom NetXtreme BCM5722 Gigabit (94309)'
     class      = network
     subclass   = ethernet
 bge1@pci0:2:0:0:        class=0x020000 card=0x020f1028 chip=0x165a14e4
 rev=0x00 hdr=0x00
     vendor     = 'Broadcom Corporation'
     device     = 'Broadcom NetXtreme BCM5722 Gigabit (94309)'
     class      = network
     subclass   = ethernet
 vgapci0@pci0:10:7:0:    class=0x030000 card=0x020f1028 chip=0x515e1002
 rev=0x02 hdr=0x00
     vendor     = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.'
     device     = 'Radeon ES1000 (Radeon ES1000)'
     class      = display
     subclass   = VGA
 -- 
 Med venlig hilsen / Best Regards
 Laust Jespersen
 http://www.ust.dk
 ======================================================================
 Viking Rule of Acquisition 1: Remember where you beached the long ship

From: Pyun YongHyeon <pyunyh@gmail.com>
To: "Laust S. Jespersen" <l@ust.dk>
Cc: yongari@freebsd.org, bug-followup@FreeBSD.org
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works
Date: Wed, 24 Nov 2010 13:48:12 -0800

 On Wed, Nov 24, 2010 at 09:10:14PM +0000, Laust S. Jespersen wrote:
 > The following reply was made to PR kern/152295; it has been noted by GNATS.
 > 
 > From: "Laust S. Jespersen" <l@ust.dk>
 > To: bug-followup@FreeBSD.org, zuborg@gmail.com
 > Cc:  
 > Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 >  on high bandwidth for BCM5784, 7.3-R works
 > Date: Wed, 24 Nov 2010 21:38:57 +0100
 > 
 >  We are seing the exact same issue on our IBM Blade servers, the
 >  upgrade to 8.2-PRE
 >  did however seem to do the trick on our dell R300 servers
 >  (which does not have the 5704 chipset)
 >  
 
 Do you mean upgrading to 8.2-PRERELEASE fix 5704 issue seen on
 7.3-RELEASE? Or You saw the same issue with 5722 on 7.3-RELEASE and
 8.2-PRERELEASE fix the 5722 issue?
 In other words, are you still seeing this issue with 5704 on
 8.2-PRERELEASE?

From: "Laust S. Jespersen" <l@ust.dk>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@FreeBSD.org
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 on high bandwidth for BCM5784, 7.3-R works
Date: Wed, 24 Nov 2010 23:37:10 +0100

 On 24-11-2010 22:48, Pyun YongHyeon wrote:
 > On Wed, Nov 24, 2010 at 09:10:14PM +0000, Laust S. Jespersen wrote:
 >> The following reply was made to PR kern/152295; it has been noted by GNATS.
 >>
 >> From: "Laust S. Jespersen"<l@ust.dk>
 >> To: bug-followup@FreeBSD.org, zuborg@gmail.com
 >> Cc:
 >> Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 >>   on high bandwidth for BCM5784, 7.3-R works
 >> Date: Wed, 24 Nov 2010 21:38:57 +0100
 >>
 >>   We are seing the exact same issue on our IBM Blade servers, the
 >>   upgrade to 8.2-PRE
 >>   did however seem to do the trick on our dell R300 servers
 >>   (which does not have the 5704 chipset)
 >>
 >
 > Do you mean upgrading to 8.2-PRERELEASE fix 5704 issue seen on
 > 7.3-RELEASE? Or You saw the same issue with 5722 on 7.3-RELEASE and
 > 8.2-PRERELEASE fix the 5722 issue?
 > In other words, are you still seeing this issue with 5704 on
 > 8.2-PRERELEASE?
 Hello Pyun :)
 
 What I meant was:
 We still saw the issue with the bge0 lockup on 8.2-PRE
 but so far only on the 5704
 we saw it on both of the chipsets on 8.1-REL so there is
 some improvement ;)
 
 I'm going onsite tonite to try to upgrade the firmware on the NIC's
 using the 2.14a update from:
 http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-53693
 (url is one line)
 
 So perhaps that will help, I'll keep you posted :)
 -- 
 Med venlig hilsen / Best Regards
 Laust Jespersen
 
 http://www.ust.dk
 ======================================================================
 Viking Rule of Acquisition 1: Remember where you beached the long ship

From: "Laust S. Jespersen" <l@ust.dk>
To: pyunyh@gmail.com
Cc: yongari@freebsd.org, bug-followup@FreeBSD.org
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 on high bandwidth for BCM5784, 7.3-R works
Date: Wed, 24 Nov 2010 23:38:23 +0100

 On 24-11-2010 22:48, Pyun YongHyeon wrote:
 > On Wed, Nov 24, 2010 at 09:10:14PM +0000, Laust S. Jespersen wrote:
 >> The following reply was made to PR kern/152295; it has been noted by GNATS.
 >>
 >> From: "Laust S. Jespersen"<l@ust.dk>
 >> To: bug-followup@FreeBSD.org, zuborg@gmail.com
 >> Cc:
 >> Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 >>   on high bandwidth for BCM5784, 7.3-R works
 >> Date: Wed, 24 Nov 2010 21:38:57 +0100
 >>
 >>   We are seing the exact same issue on our IBM Blade servers, the
 >>   upgrade to 8.2-PRE
 >>   did however seem to do the trick on our dell R300 servers
 >>   (which does not have the 5704 chipset)
 >>
 >
 > Do you mean upgrading to 8.2-PRERELEASE fix 5704 issue seen on
 > 7.3-RELEASE? Or You saw the same issue with 5722 on 7.3-RELEASE and
 > 8.2-PRERELEASE fix the 5722 issue?
 > In other words, are you still seeing this issue with 5704 on
 > 8.2-PRERELEASE?
 
 oh btw, We did not try 7.3 REL as we saw (somewhere on the interwebs)
 that other people have had issues with bge and 7.3.
 
 -- 
 Med venlig hilsen / Best Regards
 Laust Jespersen
 
 http://www.ust.dk
 ======================================================================
 Viking Rule of Acquisition 1: Remember where you beached the long ship

From: Mykola Zubach <zuborg@gmail.com>
To: pyunyh@gmail.com, yongari@freebsd.org, bug-followup@freebsd.org
Cc:  
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets
 on high bandwidth for BCM5784, 7.3-R works
Date: Thu, 25 Nov 2010 15:18:57 +0200

 --000325558202cbee980495e0725b
 Content-Type: text/plain; charset=ISO-8859-1
 
 2010/11/24 Pyun YongHyeon <pyunyh@gmail.com>
 
 > On Wed, Nov 24, 2010 at 02:10:12PM +0000, Mykola Zubach wrote:
 >
 > [...]
 >
 > >  8.1-STABLE at 2010-11-22 works stable, no problems with networking.
 > >  PR should be closed.
 > >
 >
 > Thanks for testing!
 >
 > >  Just disabled TSO feature on bge0 in order to reduce cpu usage by
 > '{swi1:
 > >  netisr 0}' kernel thread,
 > >  which was about 80% ob 600Mbit/s. POLLING is used due more efficient cpu
 > >  usage.
 > >
 >
 > It's hard to tell what is happening here without knowing details
 > but 80% for 600Mbps looks not normal. Are you seeing 80% of CPU
 > time is consumed in interrupts? Using POLLING on intelligent
 > controllers like bge(4) is not recommended. There could be some
 > situations where POLLING would give better result but most cases
 > polling just adds additional delays and system load.
 >
 > This is cpu usage without POLLING (500Mbit/s at the moment):
 
 PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
 11 root -68 - 0K 272K WAIT 2 0:23 24.56% {irq16: bge0 ehci}
 load varies in range 25-28%
 
 And this with active POLLING (same 500Mbit/s):
 
 CPU: 0.0% user, 0.0% nice, 0.9% system, 0.4% interrupt, 98.7% idle
 Mem: 160M Active, 13G Inact, 2006M Wired, 713M Cache, 1644M Buf, 119M Free
 Swap: 2048M Total, 36K Used, 2048M Free
 
 PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
 10 root 171 ki31 0K 64K CPU1 1 23.0H 100.00% {idle: cpu1}
 10 root 171 ki31 0K 64K RUN 3 23.1H 99.71% {idle: cpu3}
 10 root 171 ki31 0K 64K CPU0 0 23.9H 97.90% {idle: cpu0}
 10 root 171 ki31 0K 64K CPU2 2 24.5H 97.71% {idle: cpu2}
 3 root -8 - 0K 16K - 1 12:48 0.44% g_up
 11 root -44 - 0K 272K WAIT 0 274:27 0.24% {swi1: netisr 0}
 1172 www 44 0 48236K 34064K kqread 1 36:19 0.24% nginx
 6 root 44 - 0K 16K psleep 2 5:10 0.20% pagedaemon
 1174 www 44 0 48236K 34292K kqread 1 36:34 0.00% nginx
 1176 www 44 0 48236K 34296K kqread 2 36:24 0.00% nginx
 1173 www 44 0 48236K 34104K kqread 1 36:22 0.00% nginx
 4 root -8 - 0K 16K - 0 6:36 0.00% g_down
 19 root -8 - 0K 16K m:w1 2 5:46 0.00% g_mirror gm1
 11 root -64 - 0K 272K WAIT 0 4:28 0.00% {irq256: ahci0}
 11 root -40 - 0K 272K WAIT 0 3:58 0.00% {swi2: cambio}
 0 root 50 0 0K 96K sched 1 2:11 0.00% {swapper}
 11 root -32 - 0K 272K WAIT 0 1:57 0.00% {swi4: clock}
 
 I've applied `cpuset -l 0 -p 11` (bind 'intr' process to CPU0)
 Actually, without this cpu-binding {swi1: netisr 0} thread sometimes consume
 80%+  - load
 varies very significantly under stable bandwidth, it looks like cpu cache
 issue, not TSO one.
 
 'nginx' uses aio+sendfile, kernel is build with ZERO_SOCKETS feature.
 
 
 > >  # vmstat -i
 > >  interrupt total rate
 > >  irq16: bge0 ehci0 14426 1
 >
 > Sharing interrupts with other slow devices is not good thing. I
 > thought BCM5784 has MSI capability. Did you disable MSI?
 > Would you show me the output of "pciconf -lcbv" and dmesg output on
 > stable/8?
 >
 > >  irq23: ehci1 18671 1
 > >  cpu0: timer 19014240 2020
 > >  irq256: ahci0 6844301 727
 > >  cpu3: timer 19012976 2020
 > >  cpu2: timer 19012978 2020
 > >  cpu1: timer 19013000 2020
 > >  Total 82930592 8811
 > >
 >
 
 
 # pciconf -lcbv
 hostb0@pci0:0:0:0: class=0x060000 card=0x3318103c chip=0xd1308086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = bridge
 subclass = HOST-PCI
 cap 00[40] = unknown
 pcib1@pci0:0:3:0: class=0x060400 card=0x3318103c chip=0xd1388086 rev=0x11
 hdr=0x01
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-PCI
 cap 0d[40] = PCI Bridge card=0x3318103c
 cap 05[60] = MSI supports 2 messages, vector masks
 cap 10[90] = PCI-Express 2 root port max data 128(256) link x0(x4)
 cap 01[e0] = powerspec 3 supports D0 D3 current D0
 none0@pci0:0:8:0: class=0x088000 card=0x00000000 chip=0xd1558086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = base peripheral
 cap 10[40] = PCI-Express 2 root endpoint max data 128(128) link x0(x0)
 none1@pci0:0:8:1: class=0x088000 card=0x00000000 chip=0xd1568086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = base peripheral
 cap 10[40] = PCI-Express 2 root endpoint max data 128(128) link x0(x0)
 none2@pci0:0:8:2: class=0x088000 card=0x00000000 chip=0xd1578086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = base peripheral
 cap 10[40] = PCI-Express 2 root endpoint max data 128(128) link x0(x0)
 none3@pci0:0:8:3: class=0x088000 card=0x00000000 chip=0xd1588086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = base peripheral
 none4@pci0:0:16:0: class=0x088000 card=0x00000000 chip=0xd1508086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = base peripheral
 none5@pci0:0:16:1: class=0x088000 card=0x00000000 chip=0xd1518086 rev=0x11
 hdr=0x00
 vendor = 'Intel Corporation'
 class = base peripheral
 ehci0@pci0:0:26:0: class=0x0c0320 card=0x3118103c chip=0x3b3c8086 rev=0x05
 hdr=0x00
 vendor = 'Intel Corporation'
 class = serial bus
 subclass = USB
 bar [10] = type Memory, range 32, base 0xdfd02000, size 1024, enabled
 cap 01[50] = powerspec 2 supports D0 D3 current D0
 cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14
 cap 13[98] = PCI Advanced Features: FLR TP
 pcib2@pci0:0:28:0: class=0x060400 card=0x3318103c chip=0x3b428086 rev=0x05
 hdr=0x01
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-PCI
 cap 10[40] = PCI-Express 2 root port max data 128(128) link x0(x4)
 cap 05[80] = MSI supports 1 message
 cap 0d[90] = PCI Bridge card=0x3318103c
 cap 01[a0] = powerspec 2 supports D0 D3 current D0
 pcib3@pci0:0:28:4: class=0x060400 card=0x3318103c chip=0x3b4a8086 rev=0x05
 hdr=0x01
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-PCI
 cap 10[40] = PCI-Express 2 root port max data 128(128) link x1(x1)
 cap 05[80] = MSI supports 1 message
 cap 0d[90] = PCI Bridge card=0x3318103c
 cap 01[a0] = powerspec 2 supports D0 D3 current D0
 pcib4@pci0:0:28:5: class=0x060400 card=0x3318103c chip=0x3b4c8086 rev=0x05
 hdr=0x01
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-PCI
 cap 10[40] = PCI-Express 2 root port max data 128(128) link x1(x1)
 cap 05[80] = MSI supports 1 message
 cap 0d[90] = PCI Bridge card=0x3318103c
 cap 01[a0] = powerspec 2 supports D0 D3 current D0
 pcib5@pci0:0:28:6: class=0x060400 card=0x3318103c chip=0x3b4e8086 rev=0x05
 hdr=0x01
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-PCI
 cap 10[40] = PCI-Express 2 root port max data 128(128) link x1(x1)
 cap 05[80] = MSI supports 1 message
 cap 0d[90] = PCI Bridge card=0x3318103c
 cap 01[a0] = powerspec 2 supports D0 D3 current D0
 pcib6@pci0:0:28:7: class=0x060400 card=0x3318103c chip=0x3b508086 rev=0x05
 hdr=0x01
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-PCI
 cap 10[40] = PCI-Express 2 root port max data 128(128) link x0(x1)
 cap 05[80] = MSI supports 1 message
 cap 0d[90] = PCI Bridge card=0x3318103c
 cap 01[a0] = powerspec 2 supports D0 D3 current D0
 ehci1@pci0:0:29:0: class=0x0c0320 card=0x3118103c chip=0x3b348086 rev=0x05
 hdr=0x00
 vendor = 'Intel Corporation'
 class = serial bus
 subclass = USB
 bar [10] = type Memory, range 32, base 0xdfd02400, size 1024, enabled
 cap 01[50] = powerspec 2 supports D0 D3 current D0
 cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14
 cap 13[98] = PCI Advanced Features: FLR TP
 pcib7@pci0:0:30:0: class=0x060401 card=0x3318103c chip=0x244e8086 rev=0xa5
 hdr=0x01
 vendor = 'Intel Corporation'
 device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI
 Bridge'
 class = bridge
 subclass = PCI-PCI
 cap 0d[50] = PCI Bridge card=0x3318103c
 isab0@pci0:0:31:0: class=0x060100 card=0x3118103c chip=0x3b148086 rev=0x05
 hdr=0x00
 vendor = 'Intel Corporation'
 class = bridge
 subclass = PCI-ISA
 cap 09[e0] = vendor (length 16) Intel cap 1 version 1
 ahci0@pci0:0:31:2: class=0x010601 card=0x3118103c chip=0x3b228086 rev=0x05
 hdr=0x00
 vendor = 'Intel Corporation'
 device = 'IBEX AHCI Controller(6Port) (Intel Q57 Express)'
 class = mass storage
 subclass = SATA
 bar [10] = type I/O Port, range 32, base 0x1830, size 8, enabled
 bar [14] = type I/O Port, range 32, base 0x1824, size 4, enabled
 bar [18] = type I/O Port, range 32, base 0x1828, size 8, enabled
 bar [1c] = type I/O Port, range 32, base 0x1820, size 4, enabled
 bar [20] = type I/O Port, range 32, base 0x1800, size 32, enabled
 bar [24] = type Memory, range 32, base 0xdfd01000, size 2048, enabled
 cap 05[80] = MSI supports 1 message enabled with 1 message
 cap 01[70] = powerspec 3 supports D0 D3 current D0
 cap 12[a8] = SATA Index-Data Pair
 cap 13[b0] = PCI Advanced Features: FLR TP
 ichsmb0@pci0:0:31:3: class=0x0c0500 card=0x3318103c chip=0x3b308086 rev=0x05
 hdr=0x00
 vendor = 'Intel Corporation'
 class = serial bus
 subclass = SMBus
 bar [10] = type Memory, range 64, base 0xdfd02800, size 256, enabled
 bar [20] = type I/O Port, range 32, base 0x1840, size 32, enabled
 bge0@pci0:32:0:0: class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10
 hdr=0x00
 vendor = 'Broadcom Corporation'
 class = network
 subclass = ethernet
 bar [10] = type Memory, range 64, base 0xdf900000, size 65536, enabled
 cap 01[48] = powerspec 3 supports D0 D3 current D0
 cap 03[40] = VPD
 cap 09[60] = vendor (length 108)
 cap 05[50] = MSI supports 1 message, 64 bit
 cap 10[cc] = PCI-Express 2 endpoint max data 128(256) link x1(x1)
 bge1@pci0:34:0:0: class=0x020000 card=0x705d103c chip=0x165b14e4 rev=0x10
 hdr=0x00
 vendor = 'Broadcom Corporation'
 class = network
 subclass = ethernet
 bar [10] = type Memory, range 64, base 0xdfa00000, size 65536, enabled
 cap 01[48] = powerspec 3 supports D0 D3 current D0
 cap 03[40] = VPD
 cap 09[60] = vendor (length 108)
 cap 05[50] = MSI supports 1 message, 64 bit
 cap 10[cc] = PCI-Express 2 endpoint max data 128(256) link x1(x1)
 vgapci0@pci0:36:0:0: class=0x030000 card=0x31fa103c chip=0x0522102b rev=0x02
 hdr=0x00
 vendor = 'Matrox Electronic Systems Ltd.'
 device = 'Matrox G200e (ServerEngines) - English (G200e)'
 class = display
 subclass = VGA
 bar [10] = type Prefetchable Memory, range 32, base 0xde000000, size
 16777216, enabled
 bar [14] = type Memory, range 32, base 0xdf800000, size 16384, enabled
 bar [18] = type Memory, range 32, base 0xdf000000, size 8388608, enabled
 cap 01[dc] = powerspec 2 supports D0 D3 current D0
 cap 10[e4] = PCI-Express 1 legacy endpoint max data 128(128) link x1(x1)
 cap 05[54] = MSI supports 1 message
 
 # dmesg
 Copyright (c) 1992-2010 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 The Regents of the University of California. All rights reserved.
 FreeBSD is a registered trademark of The FreeBSD Foundation.
 FreeBSD 8.1-STABLE #1: Mon Nov 22 16:50:25 UTC 2010
 root@PRFT012:/usr/obj/usr/src/sys/Z-AMD64 amd64
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Xeon(R) CPU X3430 @ 2.40GHz (2394.00-MHz K8-class CPU)
 Origin = "GenuineIntel" Id = 0x106e5 Family = 6 Model = 1e Stepping = 5
 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 Features2=0x98e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT>
 AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
 AMD Features2=0x1<LAHF>
 TSC: P-state invariant
 real memory = 17179869184 (16384 MB)
 avail memory = 16546377728 (15779 MB)
 ACPI APIC Table: <HP ProLiant>
 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 FreeBSD/SMP: 1 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID: 0
 cpu1 (AP): APIC ID: 2
 cpu2 (AP): APIC ID: 4
 cpu3 (AP): APIC ID: 6
 ioapic0 <Version 2.0> irqs 0-23 on motherboard
 netisr_init: forcing maxthreads to 1 and bindthreads to 0 for device polling
 kbd1 at kbdmux0
 acpi0: <HP ProLiant> on motherboard
 acpi0: [ITHREAD]
 acpi0: Power Button (fixed)
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 cpu2: <ACPI CPU> on acpi0
 cpu3: <ACPI CPU> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 pcib1: <ACPI PCI-PCI bridge> irq 16 at device 3.0 on pci0
 pci1: <ACPI PCI bus> on pcib1
 pci0: <base peripheral> at device 8.0 (no driver attached)
 pci0: <base peripheral> at device 8.1 (no driver attached)
 pci0: <base peripheral> at device 8.2 (no driver attached)
 pci0: <base peripheral> at device 8.3 (no driver attached)
 pci0: <base peripheral> at device 16.0 (no driver attached)
 pci0: <base peripheral> at device 16.1 (no driver attached)
 ehci0: <Intel PCH USB 2.0 controller USB-B> mem 0xdfd02000-0xdfd023ff irq 16
 at device 26.0 on pci0
 ehci0: [ITHREAD]
 usbus0: EHCI version 1.0
 usbus0: <Intel PCH USB 2.0 controller USB-B> on ehci0
 pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
 pci16: <ACPI PCI bus> on pcib2
 pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0
 pci32: <ACPI PCI bus> on pcib3
 bge0: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem
 0xdf900000-0xdf90ffff irq 16 at device 0.0 on pci32
 bge0: CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841; PCI-E
 miibus0: <MII bus> on bge0
 brgphy0: <BCM5784 10/100/1000baseTX PHY> PHY 1 on miibus0
 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
 1000baseT-FDX, auto
 bge0: Ethernet address: d4:85:64:38:fe:ee
 bge0: [ITHREAD]
 pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.5 on pci0
 pci34: <ACPI PCI bus> on pcib4
 bge1: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem
 0xdfa00000-0xdfa0ffff irq 17 at device 0.0 on pci34
 bge1: CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841; PCI-E
 miibus1: <MII bus> on bge1
 brgphy1: <BCM5784 10/100/1000baseTX PHY> PHY 1 on miibus1
 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
 1000baseT-FDX, auto
 bge1: Ethernet address: d4:85:64:38:fe:ef
 bge1: [ITHREAD]
 pcib5: <ACPI PCI-PCI bridge> irq 18 at device 28.6 on pci0
 pci36: <ACPI PCI bus> on pcib5
 vgapci0: <VGA-compatible display> mem
 0xde000000-0xdeffffff,0xdf800000-0xdf803fff,0xdf000000-0xdf7fffff irq 18 at
 device 0.0 on pci36
 pcib6: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0
 pci38: <ACPI PCI bus> on pcib6
 ehci1: <Intel PCH USB 2.0 controller USB-A> mem 0xdfd02400-0xdfd027ff irq 23
 at device 29.0 on pci0
 ehci1: [ITHREAD]
 usbus1: EHCI version 1.0
 usbus1: <Intel PCH USB 2.0 controller USB-A> on ehci1
 pcib7: <PCI-PCI bridge> at device 30.0 on pci0
 pci48: <PCI bus> on pcib7
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 ahci0: <Intel 5 Series/3400 Series AHCI SATA controller> port
 0x1830-0x1837,0x1824-0x1827,0x1828-0x182f,0x1820-0x1823,0x1800-0x181f mem
 0xdfd01000-0xdfd017ff irq 18 at device 31.2 on pci0
 ahci0: [ITHREAD]
 ahci0: AHCI v1.30 with 6 3Gbps ports, Port Multiplier supported
 ahcich0: <AHCI channel> at channel 0 on ahci0
 ahcich0: [ITHREAD]
 ahcich1: <AHCI channel> at channel 1 on ahci0
 ahcich1: [ITHREAD]
 ahcich2: <AHCI channel> at channel 2 on ahci0
 ahcich2: [ITHREAD]
 ahcich3: <AHCI channel> at channel 3 on ahci0
 ahcich3: [ITHREAD]
 ahcich4: <AHCI channel> at channel 4 on ahci0
 ahcich4: [ITHREAD]
 ahcich5: <AHCI channel> at channel 5 on ahci0
 ahcich5: [ITHREAD]
 ichsmb0: <Intel PCH SMBus controller> port 0x1840-0x185f mem
 0xdfd02800-0xdfd028ff irq 18 at device 31.3 on pci0
 ichsmb0: [ITHREAD]
 smbus0: <System Management Bus> on ichsmb0
 smb0: <SMBus generic I/O> on smbus0
 acpi_button0: <Power Button> on acpi0
 acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8
 on acpi0
 Timecounter "HPET" frequency 14318180 Hz quality 900
 atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
 orm0: <ISA Option ROMs> at iomem
 0xc0000-0xc7fff,0xca000-0xcafff,0xdf000-0xdffff on isa0
 atkbd: unable to set the command byte.
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x300>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 atkbd0: [ITHREAD]
 psm0: unable to set the command byte.
 coretemp0: <CPU On-Die Thermal Sensors> on cpu0
 coretemp1: <CPU On-Die Thermal Sensors> on cpu1
 coretemp2: <CPU On-Die Thermal Sensors> on cpu2
 coretemp3: <CPU On-Die Thermal Sensors> on cpu3
 Timecounter "TSC" frequency 2394001965 Hz quality 800
 Timecounters tick every 0.976 msec
 ipfw2 initialized, divert loadable, nat loadable, rule-based forwarding
 disabled, default to accept, logging disabled
 usbus0: 480Mbps High Speed USB v2.0
 usbus1: 480Mbps High Speed USB v2.0
 ugen0.1: <Intel> at usbus0
 uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
 ugen1.1: <Intel> at usbus1
 uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
 ada0: <ST31000340NS MA0D> ATA-8 SATA 2.x device
 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
 ada0: Command Queueing enabled
 ada0: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
 ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
 ada1: <ST31000340NS MA0D> ATA-8 SATA 2.x device
 ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
 ada1: Command Queueing enabled
 ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
 ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
 ada2: <ST31000340NS MA0D> ATA-8 SATA 2.x device
 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
 ada2: Command Queueing enabled
 ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
 ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
 ada3: <ST31000340NS MA0D> ATA-8 SATA 2.x device
 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
 ada3: Command Queueing enabled
 ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
 SMP: AP CPU #3 Launched!
 SMP: AP CPU #2 Launched!
 SMP: AP CPU #1 Launched!
 GEOM_MIRROR: Device mirror/gm0 launched (3/3).
 GEOM: ada3s1: geometry does not match label (255h,63s != 16h,63s).
 GEOM: ada3s1: media size does not match label.
 GEOM_MIRROR: Device mirror/gm1 launched (4/4).
 Root mount waiting for: usbus1 usbus0
 uhub0: 2 ports with 2 removable, self powered
 uhub1: 2 ports with 2 removable, self powered
 ugen0.2: <vendor 0x8087> at usbus0
 uhub2: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on
 usbus0
 ugen1.2: <vendor 0x8087> at usbus1
 uhub3: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on
 usbus1
 Root mount waiting for: usbus1 usbus0
 uhub2: 6 ports with 6 removable, self powered
 uhub3: 8 ports with 8 removable, self powered
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 Trying to mount root from ufs:/dev/mirror/gm0a
 bge0: link state changed to UP
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 ugen1.3: <ServerEngines> at usbus1 (disconnected)
 ukbd0: at uhub3, port 1, addr 3 (disconnected)
 ugen1.3: <ServerEngines> at usbus1
 ukbd0: <ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3> on
 usbus1
 kbd2 at ukbd0
 
 --000325558202cbee980495e0725b
 Content-Type: text/html; charset=ISO-8859-1
 Content-Transfer-Encoding: quoted-printable
 
 <br><br><div class=3D"gmail_quote">2010/11/24 Pyun YongHyeon <span dir=3D"l=
 tr">&lt;<a href=3D"mailto:pyunyh@gmail.com">pyunyh@gmail.com</a>&gt;</span>=
 <br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef=
 t:1px #ccc solid;padding-left:1ex;">
 On Wed, Nov 24, 2010 at 02:10:12PM +0000, Mykola Zubach wrote:<br>
 <br>
 [...]<br>
 <div class=3D"im"><br>
 &gt; =A08.1-STABLE at 2010-11-22 works stable, no problems with networking.=
 <br>
 &gt; =A0PR should be closed.<br>
 &gt;<br>
 <br>
 </div>Thanks for testing!<br>
 <div class=3D"im"><br>
 &gt; =A0Just disabled TSO feature on bge0 in order to reduce cpu usage by &=
 #39;{swi1:<br>
 &gt; =A0netisr 0}&#39; kernel thread,<br>
 &gt; =A0which was about 80% ob 600Mbit/s. POLLING is used due more efficien=
 t cpu<br>
 &gt; =A0usage.<br>
 &gt;<br>
 <br>
 </div>It&#39;s hard to tell what is happening here without knowing details<=
 br>
 but 80% for 600Mbps looks not normal. Are you seeing 80% of CPU<br>
 time is consumed in interrupts? Using POLLING on intelligent<br>
 controllers like bge(4) is not recommended. There could be some<br>
 situations where POLLING would give better result but most cases<br>
 polling just adds additional delays and system load.<br>
 <div class=3D"im"><br></div></blockquote><div>This is cpu usage without POL=
 LING (500Mbit/s at the moment):</div><div><br></div><div>  PID USERNAME PRI=
  NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND<br>11 root     -68    -=
      0K   272K WAIT    2   0:23 24.56% {irq16: bge0 ehci}<br>
 </div><div>load varies in range 25-28%</div><div><br></div><div>And this wi=
 th active POLLING (same 500Mbit/s):</div><div><br></div><div>CPU:  0.0% use=
 r,  0.0% nice,  0.9% system,  0.4% interrupt, 98.7% idle<br>Mem: 160M Activ=
 e, 13G Inact, 2006M Wired, 713M Cache, 1644M Buf, 119M Free<br>
 Swap: 2048M Total, 36K Used, 2048M Free<br><br>  PID USERNAME PRI NICE   SI=
 ZE    RES STATE   C   TIME   WCPU COMMAND<br>   10 root     171 ki31     0K=
     64K CPU1    1  23.0H 100.00% {idle: cpu1}<br>   10 root     171 ki31   =
   0K    64K RUN     3  23.1H 99.71% {idle: cpu3}<br>
    10 root     171 ki31     0K    64K CPU0    0  23.9H 97.90% {idle: cpu0}<=
 br>   10 root     171 ki31     0K    64K CPU2    2  24.5H 97.71% {idle: cpu=
 2}<br>    3 root      -8    -     0K    16K -       1  12:48  0.44% g_up<br=
 >
    11 root     -44    -     0K   272K WAIT    0 274:27  0.24% {swi1: netisr=
  0}<br> 1172 www       44    0 48236K 34064K kqread  1  36:19  0.24% nginx<=
 br>    6 root      44    -     0K    16K psleep  2   5:10  0.20% pagedaemon=
 <br>
  1174 www       44    0 48236K 34292K kqread  1  36:34  0.00% nginx<br> 117=
 6 www       44    0 48236K 34296K kqread  2  36:24  0.00% nginx<br> 1173 ww=
 w       44    0 48236K 34104K kqread  1  36:22  0.00% nginx<br>    4 root  =
     -8    -     0K    16K -       0   6:36  0.00% g_down<br>
    19 root      -8    -     0K    16K m:w1    2   5:46  0.00% g_mirror gm1<=
 br>   11 root     -64    -     0K   272K WAIT    0   4:28  0.00% {irq256: a=
 hci0}<br>   11 root     -40    -     0K   272K WAIT    0   3:58  0.00% {swi=
 2: cambio}<br>
     0 root      50    0     0K    96K sched   1   2:11  0.00% {swapper}<br>=
    11 root     -32    -     0K   272K WAIT    0   1:57  0.00% {swi4: clock}=
 <br><br></div><div>I&#39;ve applied `cpuset -l 0 -p 11` (bind &#39;intr&#39=
 ; process to CPU0)</div>
 <div>Actually, without this cpu-binding {swi1: netisr 0} thread sometimes c=
 onsume 80%+ =A0- load</div><div>varies very significantly under stable band=
 width, it looks like cpu cache issue, not TSO one.</div><div><br></div><div=
 >
 &#39;nginx&#39; uses aio+sendfile, kernel is build with ZERO_SOCKETS featur=
 e.</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
  0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div class=3D"im">
 &gt; =A0# vmstat -i<br>
 &gt; =A0interrupt total rate<br>
 &gt; =A0irq16: bge0 ehci0 14426 1<br>
 <br>
 </div>Sharing interrupts with other slow devices is not good thing. I<br>
 thought BCM5784 has MSI capability. Did you disable MSI?<br>
 Would you show me the output of &quot;pciconf -lcbv&quot; and dmesg output =
 on<br>
 stable/8?<br>
 <div><div class=3D"h5"><br>
 &gt; =A0irq23: ehci1 18671 1<br>
 &gt; =A0cpu0: timer 19014240 2020<br>
 &gt; =A0irq256: ahci0 6844301 727<br>
 &gt; =A0cpu3: timer 19012976 2020<br>
 &gt; =A0cpu2: timer 19012978 2020<br>
 &gt; =A0cpu1: timer 19013000 2020<br>
 &gt; =A0Total 82930592 8811<br>
 &gt;<br>
 </div></div></blockquote></div><div><br></div><div><br># pciconf -lcbv<br>h=
 ostb0@pci0:0:0:0:      class=3D0x060000 card=3D0x3318103c chip=3D0xd1308086=
  rev=3D0x11 hdr=3D0x00<br>    vendor     =3D &#39;Intel Corporation&#39;<br=
 >    class      =3D bridge<br>
     subclass   =3D HOST-PCI<br>    cap 00[40] =3D unknown<br>pcib1@pci0:0:3=
 :0:       class=3D0x060400 card=3D0x3318103c chip=3D0xd1388086 rev=3D0x11 h=
 dr=3D0x01<br>    vendor     =3D &#39;Intel Corporation&#39;<br>    class   =
    =3D bridge<br>
     subclass   =3D PCI-PCI<br>    cap 0d[40] =3D PCI Bridge card=3D0x331810=
 3c<br>    cap 05[60] =3D MSI supports 2 messages, vector masks <br>    cap =
 10[90] =3D PCI-Express 2 root port max data 128(256) link x0(x4)<br>    cap=
  01[e0] =3D powerspec 3  supports D0 D3  current D0<br>
 none0@pci0:0:8:0:       class=3D0x088000 card=3D0x00000000 chip=3D0xd155808=
 6 rev=3D0x11 hdr=3D0x00<br>    vendor     =3D &#39;Intel Corporation&#39;<b=
 r>    class      =3D base peripheral<br>    cap 10[40] =3D PCI-Express 2 ro=
 ot endpoint max data 128(128) link x0(x0)<br>
 none1@pci0:0:8:1:       class=3D0x088000 card=3D0x00000000 chip=3D0xd156808=
 6 rev=3D0x11 hdr=3D0x00<br>    vendor     =3D &#39;Intel Corporation&#39;<b=
 r>    class      =3D base peripheral<br>    cap 10[40] =3D PCI-Express 2 ro=
 ot endpoint max data 128(128) link x0(x0)<br>
 none2@pci0:0:8:2:       class=3D0x088000 card=3D0x00000000 chip=3D0xd157808=
 6 rev=3D0x11 hdr=3D0x00<br>    vendor     =3D &#39;Intel Corporation&#39;<b=
 r>    class      =3D base peripheral<br>    cap 10[40] =3D PCI-Express 2 ro=
 ot endpoint max data 128(128) link x0(x0)<br>
 none3@pci0:0:8:3:       class=3D0x088000 card=3D0x00000000 chip=3D0xd158808=
 6 rev=3D0x11 hdr=3D0x00<br>    vendor     =3D &#39;Intel Corporation&#39;<b=
 r>    class      =3D base peripheral<br>none4@pci0:0:16:0:      class=3D0x0=
 88000 card=3D0x00000000 chip=3D0xd1508086 rev=3D0x11 hdr=3D0x00<br>
     vendor     =3D &#39;Intel Corporation&#39;<br>    class      =3D base p=
 eripheral<br>none5@pci0:0:16:1:      class=3D0x088000 card=3D0x00000000 chi=
 p=3D0xd1518086 rev=3D0x11 hdr=3D0x00<br>    vendor     =3D &#39;Intel Corpo=
 ration&#39;<br>
     class      =3D base peripheral<br>ehci0@pci0:0:26:0:      class=3D0x0c0=
 320 card=3D0x3118103c chip=3D0x3b3c8086 rev=3D0x05 hdr=3D0x00<br>    vendor=
      =3D &#39;Intel Corporation&#39;<br>    class      =3D serial bus<br>  =
   subclass   =3D USB<br>
     bar   [10] =3D type Memory, range 32, base 0xdfd02000, size 1024, enabl=
 ed<br>    cap 01[50] =3D powerspec 2  supports D0 D3  current D0<br>    cap=
  0a[58] =3D EHCI Debug Port at offset 0xa0 in map 0x14<br>    cap 13[98] =
 =3D PCI Advanced Features: FLR TP<br>
 pcib2@pci0:0:28:0:      class=3D0x060400 card=3D0x3318103c chip=3D0x3b42808=
 6 rev=3D0x05 hdr=3D0x01<br>    vendor     =3D &#39;Intel Corporation&#39;<b=
 r>    class      =3D bridge<br>    subclass   =3D PCI-PCI<br>    cap 10[40]=
  =3D PCI-Express 2 root port max data 128(128) link x0(x4)<br>
     cap 05[80] =3D MSI supports 1 message <br>    cap 0d[90] =3D PCI Bridge=
  card=3D0x3318103c<br>    cap 01[a0] =3D powerspec 2  supports D0 D3  curre=
 nt D0<br>pcib3@pci0:0:28:4:      class=3D0x060400 card=3D0x3318103c chip=3D=
 0x3b4a8086 rev=3D0x05 hdr=3D0x01<br>
     vendor     =3D &#39;Intel Corporation&#39;<br>    class      =3D bridge=
 <br>    subclass   =3D PCI-PCI<br>    cap 10[40] =3D PCI-Express 2 root por=
 t max data 128(128) link x1(x1)<br>    cap 05[80] =3D MSI supports 1 messag=
 e <br>
     cap 0d[90] =3D PCI Bridge card=3D0x3318103c<br>    cap 01[a0] =3D power=
 spec 2  supports D0 D3  current D0<br>pcib4@pci0:0:28:5:      class=3D0x060=
 400 card=3D0x3318103c chip=3D0x3b4c8086 rev=3D0x05 hdr=3D0x01<br>    vendor=
      =3D &#39;Intel Corporation&#39;<br>
     class      =3D bridge<br>    subclass   =3D PCI-PCI<br>    cap 10[40] =
 =3D PCI-Express 2 root port max data 128(128) link x1(x1)<br>    cap 05[80]=
  =3D MSI supports 1 message <br>    cap 0d[90] =3D PCI Bridge card=3D0x3318=
 103c<br>    cap 01[a0] =3D powerspec 2  supports D0 D3  current D0<br>
 pcib5@pci0:0:28:6:      class=3D0x060400 card=3D0x3318103c chip=3D0x3b4e808=
 6 rev=3D0x05 hdr=3D0x01<br>    vendor     =3D &#39;Intel Corporation&#39;<b=
 r>    class      =3D bridge<br>    subclass   =3D PCI-PCI<br>    cap 10[40]=
  =3D PCI-Express 2 root port max data 128(128) link x1(x1)<br>
     cap 05[80] =3D MSI supports 1 message <br>    cap 0d[90] =3D PCI Bridge=
  card=3D0x3318103c<br>    cap 01[a0] =3D powerspec 2  supports D0 D3  curre=
 nt D0<br>pcib6@pci0:0:28:7:      class=3D0x060400 card=3D0x3318103c chip=3D=
 0x3b508086 rev=3D0x05 hdr=3D0x01<br>
     vendor     =3D &#39;Intel Corporation&#39;<br>    class      =3D bridge=
 <br>    subclass   =3D PCI-PCI<br>    cap 10[40] =3D PCI-Express 2 root por=
 t max data 128(128) link x0(x1)<br>    cap 05[80] =3D MSI supports 1 messag=
 e <br>
     cap 0d[90] =3D PCI Bridge card=3D0x3318103c<br>    cap 01[a0] =3D power=
 spec 2  supports D0 D3  current D0<br>ehci1@pci0:0:29:0:      class=3D0x0c0=
 320 card=3D0x3118103c chip=3D0x3b348086 rev=3D0x05 hdr=3D0x00<br>    vendor=
      =3D &#39;Intel Corporation&#39;<br>
     class      =3D serial bus<br>    subclass   =3D USB<br>    bar   [10] =
 =3D type Memory, range 32, base 0xdfd02400, size 1024, enabled<br>    cap 0=
 1[50] =3D powerspec 2  supports D0 D3  current D0<br>    cap 0a[58] =3D EHC=
 I Debug Port at offset 0xa0 in map 0x14<br>
     cap 13[98] =3D PCI Advanced Features: FLR TP<br>pcib7@pci0:0:30:0:     =
  class=3D0x060401 card=3D0x3318103c chip=3D0x244e8086 rev=3D0xa5 hdr=3D0x01=
 <br>    vendor     =3D &#39;Intel Corporation&#39;<br>    device     =3D &#=
 39;82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge&#3=
 9;<br>
     class      =3D bridge<br>    subclass   =3D PCI-PCI<br>    cap 0d[50] =
 =3D PCI Bridge card=3D0x3318103c<br>isab0@pci0:0:31:0:      class=3D0x06010=
 0 card=3D0x3118103c chip=3D0x3b148086 rev=3D0x05 hdr=3D0x00<br>    vendor  =
    =3D &#39;Intel Corporation&#39;<br>
     class      =3D bridge<br>    subclass   =3D PCI-ISA<br>    cap 09[e0] =
 =3D vendor (length 16) Intel cap 1 version 1<br>ahci0@pci0:0:31:2:      cla=
 ss=3D0x010601 card=3D0x3118103c chip=3D0x3b228086 rev=3D0x05 hdr=3D0x00<br>=
     vendor     =3D &#39;Intel Corporation&#39;<br>
     device     =3D &#39;IBEX AHCI Controller(6Port) (Intel Q57 Express)&#39=
 ;<br>    class      =3D mass storage<br>    subclass   =3D SATA<br>    bar =
   [10] =3D type I/O Port, range 32, base 0x1830, size  8, enabled<br>    ba=
 r   [14] =3D type I/O Port, range 32, base 0x1824, size  4, enabled<br>
     bar   [18] =3D type I/O Port, range 32, base 0x1828, size  8, enabled<b=
 r>    bar   [1c] =3D type I/O Port, range 32, base 0x1820, size  4, enabled=
 <br>    bar   [20] =3D type I/O Port, range 32, base 0x1800, size 32, enabl=
 ed<br>
     bar   [24] =3D type Memory, range 32, base 0xdfd01000, size 2048, enabl=
 ed<br>    cap 05[80] =3D MSI supports 1 message enabled with 1 message<br> =
    cap 01[70] =3D powerspec 3  supports D0 D3  current D0<br>    cap 12[a8]=
  =3D SATA Index-Data Pair<br>
     cap 13[b0] =3D PCI Advanced Features: FLR TP<br>ichsmb0@pci0:0:31:3:   =
  class=3D0x0c0500 card=3D0x3318103c chip=3D0x3b308086 rev=3D0x05 hdr=3D0x00=
 <br>    vendor     =3D &#39;Intel Corporation&#39;<br>    class      =3D se=
 rial bus<br>
     subclass   =3D SMBus<br>    bar   [10] =3D type Memory, range 64, base =
 0xdfd02800, size 256, enabled<br>    bar   [20] =3D type I/O Port, range 32=
 , base 0x1840, size 32, enabled<br>bge0@pci0:32:0:0:       class=3D0x020000=
  card=3D0x705d103c chip=3D0x165b14e4 rev=3D0x10 hdr=3D0x00<br>
     vendor     =3D &#39;Broadcom Corporation&#39;<br>    class      =3D net=
 work<br>    subclass   =3D ethernet<br>    bar   [10] =3D type Memory, rang=
 e 64, base 0xdf900000, size 65536, enabled<br>    cap 01[48] =3D powerspec =
 3  supports D0 D3  current D0<br>
     cap 03[40] =3D VPD<br>    cap 09[60] =3D vendor (length 108)<br>    cap=
  05[50] =3D MSI supports 1 message, 64 bit <br>    cap 10[cc] =3D PCI-Expre=
 ss 2 endpoint max data 128(256) link x1(x1)<br>bge1@pci0:34:0:0:       clas=
 s=3D0x020000 card=3D0x705d103c chip=3D0x165b14e4 rev=3D0x10 hdr=3D0x00<br>
     vendor     =3D &#39;Broadcom Corporation&#39;<br>    class      =3D net=
 work<br>    subclass   =3D ethernet<br>    bar   [10] =3D type Memory, rang=
 e 64, base 0xdfa00000, size 65536, enabled<br>    cap 01[48] =3D powerspec =
 3  supports D0 D3  current D0<br>
     cap 03[40] =3D VPD<br>    cap 09[60] =3D vendor (length 108)<br>    cap=
  05[50] =3D MSI supports 1 message, 64 bit <br>    cap 10[cc] =3D PCI-Expre=
 ss 2 endpoint max data 128(256) link x1(x1)<br>vgapci0@pci0:36:0:0:    clas=
 s=3D0x030000 card=3D0x31fa103c chip=3D0x0522102b rev=3D0x02 hdr=3D0x00<br>
     vendor     =3D &#39;Matrox Electronic Systems Ltd.&#39;<br>    device  =
    =3D &#39;Matrox G200e (ServerEngines) - English (G200e)&#39;<br>    clas=
 s      =3D display<br>    subclass   =3D VGA<br>    bar   [10] =3D type Pre=
 fetchable Memory, range 32, base 0xde000000, size 16777216, enabled<br>
     bar   [14] =3D type Memory, range 32, base 0xdf800000, size 16384, enab=
 led<br>    bar   [18] =3D type Memory, range 32, base 0xdf000000, size 8388=
 608, enabled<br>    cap 01[dc] =3D powerspec 2  supports D0 D3  current D0<=
 br>
     cap 10[e4] =3D PCI-Express 1 legacy endpoint max data 128(128) link x1(=
 x1)<br>    cap 05[54] =3D MSI supports 1 message <br><br></div><div># dmesg=
 =A0<br>Copyright (c) 1992-2010 The FreeBSD Project.<br>Copyright (c) 1979, =
 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994<br>
         The Regents of the University of California. All rights reserved.<b=
 r>FreeBSD is a registered trademark of The FreeBSD Foundation.<br>FreeBSD 8=
 .1-STABLE #1: Mon Nov 22 16:50:25 UTC 2010<br>    root@PRFT012:/usr/obj/usr=
 /src/sys/Z-AMD64 amd64<br>
 Timecounter &quot;i8254&quot; frequency 1193182 Hz quality 0<br>CPU: Intel(=
 R) Xeon(R) CPU           X3430  @ 2.40GHz (2394.00-MHz K8-class CPU)<br>  O=
 rigin =3D &quot;GenuineIntel&quot;  Id =3D 0x106e5  Family =3D 6  Model =3D=
  1e  Stepping =3D 5<br>
   Features=3D0xbfebfbff&lt;FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR=
 ,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE&gt=
 ;<br>  Features2=3D0x98e3fd&lt;SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3=
 ,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT&gt;<br>
   AMD Features=3D0x28100800&lt;SYSCALL,NX,RDTSCP,LM&gt;<br>  AMD Features2=
 =3D0x1&lt;LAHF&gt;<br>  TSC: P-state invariant<br>real memory  =3D 17179869=
 184 (16384 MB)<br>avail memory =3D 16546377728 (15779 MB)<br>ACPI APIC Tabl=
 e: &lt;HP     ProLiant&gt;<br>
 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs<br>FreeBSD/SMP: 1 packa=
 ge(s) x 4 core(s)<br> cpu0 (BSP): APIC ID:  0<br> cpu1 (AP): APIC ID:  2<br=
 > cpu2 (AP): APIC ID:  4<br> cpu3 (AP): APIC ID:  6<br>ioapic0 &lt;Version =
 2.0&gt; irqs 0-23 on motherboard<br>
 netisr_init: forcing maxthreads to 1 and bindthreads to 0 for device pollin=
 g<br>kbd1 at kbdmux0<br>acpi0: &lt;HP ProLiant&gt; on motherboard<br>acpi0:=
  [ITHREAD]<br>acpi0: Power Button (fixed)<br>Timecounter &quot;ACPI-fast&qu=
 ot; frequency 3579545 Hz quality 1000<br>
 acpi_timer0: &lt;24-bit timer at 3.579545MHz&gt; port 0x1008-0x100b on acpi=
 0<br>cpu0: &lt;ACPI CPU&gt; on acpi0<br>cpu1: &lt;ACPI CPU&gt; on acpi0<br>=
 cpu2: &lt;ACPI CPU&gt; on acpi0<br>cpu3: &lt;ACPI CPU&gt; on acpi0<br>pcib0=
 : &lt;ACPI Host-PCI bridge&gt; port 0xcf8-0xcff on acpi0<br>
 pci0: &lt;ACPI PCI bus&gt; on pcib0<br>pcib1: &lt;ACPI PCI-PCI bridge&gt; i=
 rq 16 at device 3.0 on pci0<br>pci1: &lt;ACPI PCI bus&gt; on pcib1<br>pci0:=
  &lt;base peripheral&gt; at device 8.0 (no driver attached)<br>pci0: &lt;ba=
 se peripheral&gt; at device 8.1 (no driver attached)<br>
 pci0: &lt;base peripheral&gt; at device 8.2 (no driver attached)<br>pci0: &=
 lt;base peripheral&gt; at device 8.3 (no driver attached)<br>pci0: &lt;base=
  peripheral&gt; at device 16.0 (no driver attached)<br>pci0: &lt;base perip=
 heral&gt; at device 16.1 (no driver attached)<br>
 ehci0: &lt;Intel PCH USB 2.0 controller USB-B&gt; mem 0xdfd02000-0xdfd023ff=
  irq 16 at device 26.0 on pci0<br>ehci0: [ITHREAD]<br>usbus0: EHCI version =
 1.0<br>usbus0: &lt;Intel PCH USB 2.0 controller USB-B&gt; on ehci0<br>pcib2=
 : &lt;ACPI PCI-PCI bridge&gt; irq 17 at device 28.0 on pci0<br>
 pci16: &lt;ACPI PCI bus&gt; on pcib2<br>pcib3: &lt;ACPI PCI-PCI bridge&gt; =
 irq 17 at device 28.4 on pci0<br>pci32: &lt;ACPI PCI bus&gt; on pcib3<br>bg=
 e0: &lt;HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100&gt; mem =
 0xdf900000-0xdf90ffff irq 16 at device 0.0 on pci32<br>
 bge0: CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841; PCI-E<br>miibu=
 s0: &lt;MII bus&gt; on bge0<br>brgphy0: &lt;BCM5784 10/100/1000baseTX PHY&g=
 t; PHY 1 on miibus0<br>brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX=
 -FDX, 1000baseT, 1000baseT-FDX, auto<br>
 bge0: Ethernet address: d4:85:64:38:fe:ee<br>bge0: [ITHREAD]<br>pcib4: &lt;=
 ACPI PCI-PCI bridge&gt; irq 16 at device 28.5 on pci0<br>pci34: &lt;ACPI PC=
 I bus&gt; on pcib4<br>bge1: &lt;HP NC107i PCIe Gigabit Server Adapter, ASIC=
  rev. 0x5784100&gt; mem 0xdfa00000-0xdfa0ffff irq 17 at device 0.0 on pci34=
 <br>
 bge1: CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841; PCI-E<br>miibu=
 s1: &lt;MII bus&gt; on bge1<br>brgphy1: &lt;BCM5784 10/100/1000baseTX PHY&g=
 t; PHY 1 on miibus1<br>brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX=
 -FDX, 1000baseT, 1000baseT-FDX, auto<br>
 bge1: Ethernet address: d4:85:64:38:fe:ef<br>bge1: [ITHREAD]<br>pcib5: &lt;=
 ACPI PCI-PCI bridge&gt; irq 18 at device 28.6 on pci0<br>pci36: &lt;ACPI PC=
 I bus&gt; on pcib5<br>vgapci0: &lt;VGA-compatible display&gt; mem 0xde00000=
 0-0xdeffffff,0xdf800000-0xdf803fff,0xdf000000-0xdf7fffff irq 18 at device 0=
 .0 on pci36<br>
 pcib6: &lt;ACPI PCI-PCI bridge&gt; irq 19 at device 28.7 on pci0<br>pci38: =
 &lt;ACPI PCI bus&gt; on pcib6<br>ehci1: &lt;Intel PCH USB 2.0 controller US=
 B-A&gt; mem 0xdfd02400-0xdfd027ff irq 23 at device 29.0 on pci0<br>ehci1: [=
 ITHREAD]<br>
 usbus1: EHCI version 1.0<br>usbus1: &lt;Intel PCH USB 2.0 controller USB-A&=
 gt; on ehci1<br>pcib7: &lt;PCI-PCI bridge&gt; at device 30.0 on pci0<br>pci=
 48: &lt;PCI bus&gt; on pcib7<br>isab0: &lt;PCI-ISA bridge&gt; at device 31.=
 0 on pci0<br>
 isa0: &lt;ISA bus&gt; on isab0<br>ahci0: &lt;Intel 5 Series/3400 Series AHC=
 I SATA controller&gt; port 0x1830-0x1837,0x1824-0x1827,0x1828-0x182f,0x1820=
 -0x1823,0x1800-0x181f mem 0xdfd01000-0xdfd017ff irq 18 at device 31.2 on pc=
 i0<br>
 ahci0: [ITHREAD]<br>ahci0: AHCI v1.30 with 6 3Gbps ports, Port Multiplier s=
 upported<br>ahcich0: &lt;AHCI channel&gt; at channel 0 on ahci0<br>ahcich0:=
  [ITHREAD]<br>ahcich1: &lt;AHCI channel&gt; at channel 1 on ahci0<br>ahcich=
 1: [ITHREAD]<br>
 ahcich2: &lt;AHCI channel&gt; at channel 2 on ahci0<br>ahcich2: [ITHREAD]<b=
 r>ahcich3: &lt;AHCI channel&gt; at channel 3 on ahci0<br>ahcich3: [ITHREAD]=
 <br>ahcich4: &lt;AHCI channel&gt; at channel 4 on ahci0<br>ahcich4: [ITHREA=
 D]<br>
 ahcich5: &lt;AHCI channel&gt; at channel 5 on ahci0<br>ahcich5: [ITHREAD]<b=
 r>ichsmb0: &lt;Intel PCH SMBus controller&gt; port 0x1840-0x185f mem 0xdfd0=
 2800-0xdfd028ff irq 18 at device 31.3 on pci0<br>ichsmb0: [ITHREAD]<br>
 smbus0: &lt;System Management Bus&gt; on ichsmb0<br>smb0: &lt;SMBus generic=
  I/O&gt; on smbus0<br>acpi_button0: &lt;Power Button&gt; on acpi0<br>acpi_h=
 pet0: &lt;High Precision Event Timer&gt; iomem 0xfed00000-0xfed003ff irq 0,=
 8 on acpi0<br>
 Timecounter &quot;HPET&quot; frequency 14318180 Hz quality 900<br>atrtc0: &=
 lt;AT realtime clock&gt; port 0x70-0x71 on acpi0<br>orm0: &lt;ISA Option RO=
 Ms&gt; at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdf000-0xdffff on isa0<br>
 atkbd: unable to set the command byte.<br>sc0: &lt;System console&gt; at fl=
 ags 0x100 on isa0<br>sc0: VGA &lt;16 virtual consoles, flags=3D0x300&gt;<br=
 >vga0: &lt;Generic ISA VGA&gt; at port 0x3c0-0x3df iomem 0xa0000-0xbffff on=
  isa0<br>
 atkbdc0: &lt;Keyboard controller (i8042)&gt; at port 0x60,0x64 on isa0<br>a=
 tkbd0: &lt;AT Keyboard&gt; irq 1 on atkbdc0<br>kbd0 at atkbd0<br>atkbd0: [G=
 IANT-LOCKED]<br>atkbd0: [ITHREAD]<br>psm0: unable to set the command byte.<=
 br>
 coretemp0: &lt;CPU On-Die Thermal Sensors&gt; on cpu0<br>coretemp1: &lt;CPU=
  On-Die Thermal Sensors&gt; on cpu1<br>coretemp2: &lt;CPU On-Die Thermal Se=
 nsors&gt; on cpu2<br>coretemp3: &lt;CPU On-Die Thermal Sensors&gt; on cpu3<=
 br>
 Timecounter &quot;TSC&quot; frequency 2394001965 Hz quality 800<br>Timecoun=
 ters tick every 0.976 msec<br>ipfw2 initialized, divert loadable, nat loada=
 ble, rule-based forwarding disabled, default to accept, logging disabled<br=
 >
 usbus0: 480Mbps High Speed USB v2.0<br>usbus1: 480Mbps High Speed USB v2.0<=
 br>ugen0.1: &lt;Intel&gt; at usbus0<br>uhub0: &lt;Intel EHCI root HUB, clas=
 s 9/0, rev 2.00/1.00, addr 1&gt; on usbus0<br>ugen1.1: &lt;Intel&gt; at usb=
 us1<br>
 uhub1: &lt;Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1&gt; on usb=
 us1<br>ada0 at ahcich0 bus 0 scbus0 target 0 lun 0<br>ada0: &lt;ST31000340N=
 S MA0D&gt; ATA-8 SATA 2.x device<br>ada0: 300.000MB/s transfers (SATA 2.x, =
 UDMA6, PIO 8192bytes)<br>
 ada0: Command Queueing enabled<br>ada0: 953869MB (1953525168 512 byte secto=
 rs: 16H 63S/T 16383C)<br>ada1 at ahcich1 bus 0 scbus1 target 0 lun 0<br>ada=
 1: &lt;ST31000340NS MA0D&gt; ATA-8 SATA 2.x device<br>ada1: 300.000MB/s tra=
 nsfers (SATA 2.x, UDMA6, PIO 8192bytes)<br>
 ada1: Command Queueing enabled<br>ada1: 953869MB (1953525168 512 byte secto=
 rs: 16H 63S/T 16383C)<br>ada2 at ahcich2 bus 0 scbus2 target 0 lun 0<br>ada=
 2: &lt;ST31000340NS MA0D&gt; ATA-8 SATA 2.x device<br>ada2: 300.000MB/s tra=
 nsfers (SATA 2.x, UDMA6, PIO 8192bytes)<br>
 ada2: Command Queueing enabled<br>ada2: 953869MB (1953525168 512 byte secto=
 rs: 16H 63S/T 16383C)<br>ada3 at ahcich3 bus 0 scbus3 target 0 lun 0<br>ada=
 3: &lt;ST31000340NS MA0D&gt; ATA-8 SATA 2.x device<br>ada3: 300.000MB/s tra=
 nsfers (SATA 2.x, UDMA6, PIO 8192bytes)<br>
 ada3: Command Queueing enabled<br>ada3: 953869MB (1953525168 512 byte secto=
 rs: 16H 63S/T 16383C)<br>SMP: AP CPU #3 Launched!<br>SMP: AP CPU #2 Launche=
 d!<br>SMP: AP CPU #1 Launched!<br>GEOM_MIRROR: Device mirror/gm0 launched (=
 3/3).<br>
 GEOM: ada3s1: geometry does not match label (255h,63s !=3D 16h,63s).<br>GEO=
 M: ada3s1: media size does not match label.<br>GEOM_MIRROR: Device mirror/g=
 m1 launched (4/4).<br>Root mount waiting for: usbus1 usbus0<br>uhub0: 2 por=
 ts with 2 removable, self powered<br>
 uhub1: 2 ports with 2 removable, self powered<br>ugen0.2: &lt;vendor 0x8087=
 &gt; at usbus0<br>uhub2: &lt;vendor 0x8087 product 0x0020, class 9/0, rev 2=
 .00/0.00, addr 2&gt; on usbus0<br>ugen1.2: &lt;vendor 0x8087&gt; at usbus1<=
 br>
 uhub3: &lt;vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2&g=
 t; on usbus1<br>Root mount waiting for: usbus1 usbus0<br>uhub2: 6 ports wit=
 h 6 removable, self powered<br>uhub3: 8 ports with 8 removable, self powere=
 d<br>
 ugen1.3: &lt;ServerEngines&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB=
  Device, class 0/0, rev 1.10/0.01, addr 3&gt; on usbus1<br>kbd2 at ukbd0<br=
 >Trying to mount root from ufs:/dev/mirror/gm0a<br>bge0: link state changed=
  to UP<br>
 ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br>ukbd0: at uhub3,=
  port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngines&gt; at usbus1<=
 br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev 1.10/0.01, addr 3=
 &gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br>ugen1.3: &lt;ServerEngines&gt; at usbus1 (disconnected)<br=
 >ukbd0: at uhub3, port 1, addr 3 (disconnected)<br>ugen1.3: &lt;ServerEngin=
 es&gt; at usbus1<br>ukbd0: &lt;ServerEngines SE USB Device, class 0/0, rev =
 1.10/0.01, addr 3&gt; on usbus1<br>
 kbd2 at ukbd0<br><br></div>
 
 --000325558202cbee980495e0725b--

From: Pyun YongHyeon <pyunyh@gmail.com>
To: "Laust S. Jespersen" <l@ust.dk>
Cc: yongari@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming packets on high bandwidth for BCM5784, 7.3-R works
Date: Mon, 31 Jan 2011 15:56:41 -0800

 On Wed, Nov 24, 2010 at 11:38:23PM +0100, Laust S. Jespersen wrote:
 > On 24-11-2010 22:48, Pyun YongHyeon wrote:
 > >On Wed, Nov 24, 2010 at 09:10:14PM +0000, Laust S. Jespersen wrote:
 > >>The following reply was made to PR kern/152295; it has been noted by 
 > >>GNATS.
 > >>
 > >>From: "Laust S. Jespersen"<l@ust.dk>
 > >>To: bug-followup@FreeBSD.org, zuborg@gmail.com
 > >>Cc:
 > >>Subject: Re: kern/152295: [bge] 8.1-R driver stop to reveice incoming 
 > >>packets
 > >>  on high bandwidth for BCM5784, 7.3-R works
 > >>Date: Wed, 24 Nov 2010 21:38:57 +0100
 > >>
 > >>  We are seing the exact same issue on our IBM Blade servers, the
 > >>  upgrade to 8.2-PRE
 > >>  did however seem to do the trick on our dell R300 servers
 > >>  (which does not have the 5704 chipset)
 > >>
 > >
 > >Do you mean upgrading to 8.2-PRERELEASE fix 5704 issue seen on
 > >7.3-RELEASE? Or You saw the same issue with 5722 on 7.3-RELEASE and
 > >8.2-PRERELEASE fix the 5722 issue?
 > >In other words, are you still seeing this issue with 5704 on
 > >8.2-PRERELEASE?
 > 
 > oh btw, We did not try 7.3 REL as we saw (somewhere on the interwebs)
 > that other people have had issues with bge and 7.3.
 > 
 
 There was a bge(4) issue which triggered watchdog timeouts on
 PCI-X BCM5704 and it was resolved at the end of release cycle.
 However the issue seems to happen on systems that have more than
 4GB memory under specific PCI-X bridge controller. Your previous
 dmesg output indicates you have just 1GB memory so your issue
 could be different one.
 I think it's better to open a new PR about your issue since the
 original submitter used different controllers. Do you also see
 exactly same issue of this PR? Does TX still work? How about
 hardware MAC statistics? Does it still work?
 
 Please include the following information in the new PR. Don't
 mention BCM5722 now works on 8.2 since it's not related with the
 BCM5704 issue.
 
  - dmesg output
  - pciconf -lcbv
  - ifconfig bge0 output
  - sysctl dev.bge.0.stats output
  - If you know a way to reliably trigger the issue please include
    it.
  - Let me know the new PR number in order to grab it.
State-Changed-From-To: feedback->closed 
State-Changed-By: yongari 
State-Changed-When: Wed Feb 2 01:06:57 UTC 2011 
State-Changed-Why:  
Close. Original submitter confirms the issue was resolved in latest 
stable/8 which would be 8.2-RELEASE in very near future. 
Laust's issue was not caused by bge(4). 
Thanks for your report and testing! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152295 
>Unformatted:
