From nobody@FreeBSD.org  Sat Mar 12 19:38:49 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 30DDE1065670
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 12 Mar 2011 19:38:49 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 06ED78FC0A
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 12 Mar 2011 19:38:49 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p2CJcmXc080661
	for <freebsd-gnats-submit@FreeBSD.org>; Sat, 12 Mar 2011 19:38:48 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p2CJcmZX080660;
	Sat, 12 Mar 2011 19:38:48 GMT
	(envelope-from nobody)
Message-Id: <201103121938.p2CJcmZX080660@red.freebsd.org>
Date: Sat, 12 Mar 2011 19:38:48 GMT
From: Benjamin Lee <ben@b1c1l1.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: mfi(4) command timeout loop on boot on releng/8.2
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         155499
>Category:       kern
>Synopsis:       [mfi] mfi(4) command timeout loop on boot on releng/8.2
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    jhb
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Mar 12 19:40:10 UTC 2011
>Closed-Date:    Fri Dec 14 03:02:16 UTC 2012
>Last-Modified:  Fri Dec 14 03:02:16 UTC 2012
>Originator:     Benjamin Lee
>Release:        8.2-RELEASE
>Organization:
>Environment:
FreeBSD genesis.b1c1l1.com 8.2-RELEASE FreeBSD 8.2-RELEASE #11: Sat Mar 12 11:10:09 PST 2011     root@genesis.b1c1l1.com:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
When trying to boot an 8.2-RELEASE kernel, mfi(4) enters an infinite command timeout loop.

With MFI_DEBUG I get the following output:

mfi0: COMMAND 0xffffff80002c6440 TIMEOUT AFTER 59 SECONDS
mfi0: cm=0xffffff80002c6440 index=8 total_frame_size=52 extra_frames=0
mfi0: flags=83<MAPPED,DATAIN,Q_BUSY>
mfi0: cmd=MFI_CMD_DCMD opcode=LD_GET_LIST data_len=1032
mfi0: flags=12<SGL64,READ>
SG List:
0x1a3d000:001032
mfi0: mfi_timeout 2522 COMMAND 0xffffff80002c6440 S/G count bad 0 1032 1032 
0x1a3d000
mfi0: cm=0xffffff80002c6440 index=8 total_frame_size=52 extra_frames=0
mfi0: flags=83<MAPPED,DATAIN,Q_BUSY>
mfi0: cmd=MFI_CMD_DCMD opcode=LD_GET_LIST data_len=1032
mfi0: flags=12<SGL64,READ>
SG List:
0x1a3d000:001032

This output repeats itself every 30 seconds (with the timeout counter increasing).

This exact system does not have any problems with 8.1-RELEASE.

The card is an LSI MegaRAID SAS 8408E:

blee@genesis ~ $ pciconf -lv | grep -A4 mfi0
mfi0@pci0:6:14:0:       class=0x010400 card=0x10011000 chip=0x04111000 rev=0x00 hdr=0x00
    vendor     = 'LSI Logic (Was: Symbios Logic, NCR)'
    device     = 'MegaRAID Family RAID Controller'
    class      = mass storage
    subclass   = RAID
blee@genesis ~ $ sudo mfiutil show adapter
mfi0 Adapter:
    Product Name: MegaRAID SAS 8408E
   Serial Number: P186584707
        Firmware: 7.0.1-0081
     RAID Levels: JBOD, RAID0, RAID1, RAID5, RAID10, RAID50
  Battery Backup: present
           NVRAM: 32K
  Onboard Memory: 256M
  Minimum Stripe: 8K
  Maximum Stripe: 1M

Upgrading to the latest firmware had no effect:

blee@genesis ~ $ sudo mfiutil show firmware
mfi0 Firmware Package Version: 7.0.1-0081
mfi0 Firmware Images:
Name  Version                Date         Time      Status
BTBL  R.2.3.15               May 20 2008  16:09:27  active
BIOS  MT33                                          active
MPT1  MPTFW-01.18.172.00-IT  01/31/11     16:49:08  active
APP   1.12.310-1112          Dec 17 2010  14:54:35  active
BCON  1.1-33j-e_11-Rel       Aug 20 2009  21:16:24  active
CTLR  1.04-019A              Aug 13 2007  23:21:34  active
MPT3  MPTFW-01.18.172.00-IT  01/31/11     16:49:29  active
PCLI  01.00-011:#%00001      Oct 10 2007  16:30:36  active

>How-To-Repeat:
1) Boot 8.1-RELEASE kernel.  Observe no problems.
2) Boot 8.2-RELEASE kernel.  Kernel will not boot due to command timeout loop in mfi(4).
>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sat Mar 12 20:37:24 UTC 2011 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155499 
Responsible-Changed-From-To: freebsd-net->freebsd-bugs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sat Mar 12 20:37:45 UTC 2011 
Responsible-Changed-Why:  
er, fix braino. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155499 
State-Changed-From-To: open->feedback 
State-Changed-By: smh 
State-Changed-When: Thu Dec 13 22:55:29 UTC 2012 
State-Changed-Why:  
Could you try this with a recent 9.1 snap as mfi has undergone some 
major changes. mfi timeouts are also known to broken, they don't ever 
timeout. You could try the following patch which fixes that against 
HEAD: http://blog.multiplay.co.uk/dropzone/freebsd/zz-mfi-queue.patch 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155499 
State-Changed-From-To: feedback->closed 
State-Changed-By: eadler 
State-Changed-When: Fri Dec 14 03:02:14 UTC 2012 
State-Changed-Why:  
fixed in r227562 


Responsible-Changed-From-To: freebsd-bugs->jhb 
Responsible-Changed-By: eadler 
Responsible-Changed-When: Fri Dec 14 03:02:14 UTC 2012 
Responsible-Changed-Why:  
fixed in r227562 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155499 
>Unformatted:
