From nobody@FreeBSD.org  Tue Jan 13 05:02:07 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 7CF6F106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 13 Jan 2009 05:02:07 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 6B6398FC08
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 13 Jan 2009 05:02:07 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n0D527J0095568
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 13 Jan 2009 05:02:07 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n0D5275X095567;
	Tue, 13 Jan 2009 05:02:07 GMT
	(envelope-from nobody)
Message-Id: <200901130502.n0D5275X095567@www.freebsd.org>
Date: Tue, 13 Jan 2009 05:02:07 GMT
From: Xiuchao Wu <wuxiuchao@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: MSI must be disabled when Myricom 10Gbps Card is used on Dell PowerEdge T300 Server
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         130483
>Category:       amd64
>Synopsis:       [mxge] MSI must be disabled when Myricom 10Gbps Card is used on Dell PowerEdge T300 Server
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-amd64
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 13 05:10:01 UTC 2009
>Closed-Date:    Wed Sep 01 13:13:36 UTC 2010
>Last-Modified:  Wed Sep 01 13:13:36 UTC 2010
>Originator:     Xiuchao Wu
>Release:        FreeBSD 7.0
>Organization:
National University of Singapore
>Environment:
FreeBSD 7.0/7.1 Release
>Description:
Computer: Dell PowerEdge T300 Server
NICs: Myricom 10G-PCIE-8AL-C
OS: FreeBSD 7.0/7.1 Release

After I rebuild the kernel (to include NIC driver "mxge") and reboot, kernel panic occurs and the computer is automatically rebooted. After contacting with engineers of Myricom, they suggest to disable message signaled interrupt (MSI) in /boot/loader.conf and these NICs can work now.
"
hw.pci.enable_msix=0
hw.pci.enable_msi=0
"

However, MSI is really very important for high speed data transmission. Huge number of packets generate many interruptions. Considering that Fedora 9 works well on the same computer, it may be a bug of FreeBSD. Below is the screen when kernel panic occurs.


......
p4tcc3: <CPU Frequency Thermal Control> on cpu3
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci3: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci4: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci5: <ACPI PCI bus> on pcib3
mxge0: <Myri10G-PCIE-8A> mem 0xd8000000-0xd8ffffff, 0xdfa00000-0xdfafffff irq 16 at device 0.0 on pci5
panic: nexus_add_irq: failed
.....




Best Regards,
Xiuchao Wu (wuxiuchao@gmail.com)

>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:

From: John Baldwin <jhb@freebsd.org>
To: freebsd-amd64@freebsd.org
Cc: Xiuchao Wu <wuxiuchao@gmail.com>, freebsd-gnats-submit@freebsd.org
Subject: Re: amd64/130483: MSI must be disabled when Myricom 10Gbps Card is used on Dell PowerEdge T300 Server
Date: Tue, 20 Jan 2009 11:57:57 -0500

 How many MSI messages is mxge0 attempting to add and how many other devices in 
 your system are using MSI?  Getting a full copy of a verbose dmesg would be 
 most helpful.  'pciconf -lc' might also help answer the first question.
 
 -- 
 John Baldwin
State-Changed-From-To: open->feedback 
State-Changed-By: linimon 
State-Changed-When: Sat Mar 13 19:56:53 UTC 2010 
State-Changed-Why:  
Note that submitter feedback was requested. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=130483 
State-Changed-From-To: feedback->closed 
State-Changed-By: jh 
State-Changed-When: Wed Sep 1 13:13:36 UTC 2010 
State-Changed-Why:  
Feedback timeout. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=130483 
>Unformatted:
