From nobody@FreeBSD.org  Thu Mar  6 11:12:07 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id 731C4852
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  6 Mar 2014 11:12:07 +0000 (UTC)
Received: from cgiserv.freebsd.org (cgiserv.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 535E6278
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  6 Mar 2014 11:12:07 +0000 (UTC)
Received: from cgiserv.freebsd.org ([127.0.1.6])
	by cgiserv.freebsd.org (8.14.8/8.14.8) with ESMTP id s26BC6BF020681
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 6 Mar 2014 11:12:06 GMT
	(envelope-from nobody@cgiserv.freebsd.org)
Received: (from nobody@localhost)
	by cgiserv.freebsd.org (8.14.8/8.14.8/Submit) id s26BC61N020678;
	Thu, 6 Mar 2014 11:12:06 GMT
	(envelope-from nobody)
Message-Id: <201403061112.s26BC61N020678@cgiserv.freebsd.org>
Date: Thu, 6 Mar 2014 11:12:06 GMT
From: Borja Marcos <borjam@sarenet.es>
To: freebsd-gnats-submit@FreeBSD.org
Subject: LSI Logic Invader card fails at boot
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         187312
>Category:       kern
>Synopsis:       LSI Logic Invader card fails at boot
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    markj
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Mar 06 11:20:01 UTC 2014
>Closed-Date:    Mon Mar 10 03:05:03 UTC 2014
>Last-Modified:  Mon Mar 10 03:05:03 UTC 2014
>Originator:     Borja Marcos
>Release:        FreeBSD 10-STABLE
>Organization:
>Environment:
FreeBSD elibm 10.0-STABLE FreeBSD 10.0-STABLE #1: Tue Mar  4 09:36:48 UTC 2014     root@:/usr/obj/usr/src/sys/GENERIC  amd64

>Description:
There is some trouble with this LSI Logic RAID card:

mfi0 Adapter:
    Product Name: ServeRAID M5210e
   Serial Number: 3CJ0SG          
        Firmware: 24.0.2-0013
     RAID Levels: JBOD, RAID0, RAID1, RAID10
  Battery Backup: not present
           NVRAM: 32K
  Onboard Memory: 0M
  Minimum Stripe: 64K
  Maximum Stripe: 64K


It's an Invader series card. When booting from install media (for example, a memstick) FreeBSD detects the RAID volume (/dev/mfid0) without issues and the installation completes. There's just a warning message:

However, once the system is installed, FreeBSD is unable to boot. The loader works, the kernel starts, but it fails when trying to mount the root file system from /dev/mfid0p2.

It fails with 10-RELEASE and 10-STABLE.


The errors are:

mfi0: Adapter RESET condition is detected
mfi0: First step of FW reset initialized
mfi0: ADP_RESET_TBOLT: Hostdiag=0xa0
mfi0: First stage of reset complete, second stage initialized
mfi0: APJ ****requeue command 0xfffffe000104dfc0 index=120
mfi0: second stage of reset complete, FW is ready now.
mfi0: 11032 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/045b/1014)
mfi0: 11033 (boot + 3s/0x0020/info) - Firmware version 4.200.21-2840


I haven't copied all the errors, last time I tried it finished entering an endless loop.

Interestingly, IT WORKS With 11-CURRENT. I can successfully install 11-CURRENT and boot normally.

>How-To-Repeat:
Create a RAID 1 volume (it's the only option I have tried) and install 10-RELEASE or 10-STABLE on /dev/mfid0, standard installation, no frills.

>Fix:
It works with 11-CURRENT.

Comparing the mfi driver betwen CURRENT and STABLE I noticed that there are three files with changes,

sys/dev/mfi/mfi_pci.c
sys/dev/mfi/mfi_tbolt.c
sys/dev/mvi/mfivar.h

http://svnweb.freebsd.org/base/head/sys/dev/mfi/mfi_pci.c?revision=261535&view=markup
http://svnweb.freebsd.org/base/head/sys/dev/mfi/mfi_tbolt.c?revision=261535&view=markup
http://svnweb.freebsd.org/base/head/sys/dev/mfi/mfivar.h?revision=261535&view=markup

I copied the three files on a -STABLE source tree checked out yesterday, built a kernel and tried. The -STABLE kernel with mfi compiled with these new files works. It can boot from /dev/mfid0 and mount /dev/mfid0p2 as the root file system.

mfi0: <Invader> port 0x4f00-0x4fff mem 0x913f0000-0x913fffff,0x91400000-0x914fffff irq 34 at device 0.0 on pci22
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23 
mfi0: FW MaxCmds = 240, limiting to 128
mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c00f0
mfi0: 11019 (447416787s/0x0020/info) - Shutdown command received from host
mfi0: 11020 (boot + 10s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/045b/1014)
mfi0: 11021 (boot + 10s/0x0020/info) - Firmware version 4.200.21-2840
mfi0: 11022 (boot + 12s/0x0020/info) - Package version 24.0.2-0013
mfi0: 11023 (boot + 12s/0x0020/info) - Board Revision  00AL055
mfi0: 11024 (boot + 33s/0x0002/info) - Inserted: PD 02(e0x3e/s25)
mfi0: 11025 (boot + 33s/0x0002/info) - Inserted: PD 02(e0x3e/s25) Info: enclPd=3e, scsiType=0, portMap=00, sasAddr=4433221102000000,0000000000000000
mfi0: 11026 (boot + 33s/0x0002/info) - PD 02(e0x3e/s25) FRU is 90Y8649
mfi0: 11027 (boot + 33s/0x0002/info) - Inserted: PD 03(e0x3e/s24)
mfi0: 11028 (boot + 33s/0x0002/info) - Inserted: PD 03(e0x3e/s24) Info: enclPd=3e, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000
mfi0: 11029 (boot + 33s/0x0002/info) - PD 03(e0x3e/s24) FRU is 90Y8649
mfi0: 11030 (447416847s/0x0020/info) - Time established as 03/06/14 10:27:27; (33 seconds since power on)

mfi0: 11031 (447416924s/0x0020/info) - Time established as 03/06/14 10:28:44; (111 seconds since power on)
mfi0: 11032 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/045b/1014)
mfi0: 11033 (boot + 3s/0x0020/info) - Firmware version 4.200.21-2840
mfi0: 11034 (447417030s/0x0020/CRIT) - Controller encountered a fatal error and was reset
mfi0: 11035 (447417032s/0x0020/info) - Package version 24.0.2-0013
mfi0: 11036 (447417032s/0x0020/info) - Board Revision  00AL055
mfi0: 11037 (447417052s/0x0002/info) - Inserted: PD 02(e0x3e/s25)
mfi0: 11038 (447417052s/0x0002/info) - Inserted: PD 02(e0x3e/s25) Info: enclPd=3e, scsiType=0, portMap=00, sasAddr=4433221102000000,0000000000000000
mfi0: 11039 (447417052s/0x0002/info) - PD 02(e0x3e/s25) FRU is 90Y8649
mfi0: 11040 (447417052s/0x0002/info) - Inserted: PD 03(e0x3e/s24)
mfi0: 11041 (447417052s/0x0002/info) - Inserted: PD 03(e0x3e/s24) Info: enclPd=3e, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000
mfi0: 11042 (447417052s/0x0002/info) - PD 03(e0x3e/s24) FRU is 90Y8649
mfi0: 11043 (447417052s/0x0020/info) - Controller Hot Plug detected
mfi0: 11044 (boot + 10s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/045b/1014)

mfi0: 11045 (boot + 10s/0x0020/info) - Firmware version 4.200.21-2840
mfi0: 11046 (boot + 12s/0x0020/info) - Package version 24.0.2-0013
mfi0: 11047 (boot + 12s/0x0020/info) - Board Revision  00AL055
mfi0: 11048 (boot + 32s/0x0002/info) - Inserted: PD 02(e0x3e/s25)
mfi0: 11049 (boot + 32s/0x0002/info) - Inserted: PD 02(e0x3e/s25) Info: enclPd=3e, scsiType=0, portMap=00, sasAddr=4433221102000000,0000000000000000
mfi0: 11050 (boot + 32s/0x0002/info) - PD 02(e0x3e/s25) FRU is 90Y8649
mfi0: 11051 (boot + 32s/0x0002/info) - Inserted: PD 03(e0x3e/s24)
mfi0: 11052 (boot + 32s/0x0002/info) - Inserted: PD 03(e0x3e/s24) Info: enclPd=3e, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000
mfi0: 11053 (boot + 32s/0x0002/info) - PD 03(e0x3e/s24) FRU is 90Y8649
mfi0: 11054 (447417194s/0x0020/info) - Time established as 03/06/14 10:33:14; (32 seconds since power on)
mfi0: 11055 (447417277s/0x0020/info) - Time established as 03/06/14 10:34:37; (115 seconds since power on)

mfid0 on mfi0
mfid0: 121116MB (248045568 sectors) RAID volume (no label) is optimal
mfi0: Failed to get command
mfi0: <Invader> port 0x4f00-0x4fff mem 0x913f0000-0x913fffff,0x91400000-0x914fffff irq 34 at device 0.0 on pci22
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23 
mfi0: FW MaxCmds = 240, limiting to 128
mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c00f0
mfi0: 11056 (447417497s/0x0020/info) - Shutdown command received from host
mfi0: 11057 (boot + 10s/0x0020/info) - Firmware initialization started (PCI ID 005d/1000/045b/1014)
mfi0: 11058 (boot + 10s/0x0020/info) - Firmware version 4.200.21-2840
mfi0: 11059 (boot + 12s/0x0020/info) - Package version 24.0.2-0013
mfi0: 11060 (boot + 12s/0x0020/info) - Board Revision  00AL055
mfi0: 11061 (boot + 33s/0x0002/info) - Inserted: PD 02(e0x3e/s25)



mfi0: 11062 (boot + 33s/0x0002/info) - Inserted: PD 02(e0x3e/s25) Info: enclPd=3e, scsiType=0, portMap=00, sasAddr=4433221102000000,0000000000000000
mfi0: 11063 (boot + 33s/0x0002/info) - PD 02(e0x3e/s25) FRU is 90Y8649
mfi0: 11064 (boot + 33s/0x0002/info) - Inserted: PD 03(e0x3e/s24)
mfi0: 11065 (boot + 33s/0x0002/info) - Inserted: PD 03(e0x3e/s24) Info: enclPd=3e, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000
mfi0: 11066 (boot + 33s/0x0002/info) - PD 03(e0x3e/s24) FRU is 90Y8649
mfi0: 11067 (447417556s/0x0020/info) - Time established as 03/06/14 10:39:16; (33 seconds since power on)
mfi0: 11068 (447417632s/0x0020/info) - Time established as 03/06/14 10:40:32; (110 seconds since power on)
mfid0 on mfi0
mfid0: 121116MB (248045568 sectors) RAID volume (no label) is optimal
Trying to mount root from ufs:/dev/mfid0p2 [rw]...




There's still some problem with the driver (see  the "fatal error" message) but at least it works. I'm going to do some checks, run some heavy I/O activity to see if it can be reliable.

In case this solves the issue, could the above mentioned changes be backported to 10-STABLE?



>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->markj 
Responsible-Changed-By: markj 
Responsible-Changed-When: Fri Mar 7 14:22:07 UTC 2014 
Responsible-Changed-Why:  
I fixed the problem in HEAD. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=187312 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/187312: commit references a PR
Date: Mon, 10 Mar 2014 02:31:57 +0000 (UTC)

 Author: markj
 Date: Mon Mar 10 02:31:50 2014
 New Revision: 262967
 URL: http://svnweb.freebsd.org/changeset/base/262967
 
 Log:
   MFC r261535:
   Add support for MegaRAID Fury cards. The main change needed to boot from a
   9341-4i controller was to ensure that scatter/gather lists are ended with
   an end-of-list marker. Both the mrsas and Linux megaraid_sas drivers use
   this marker with Invader cards as well, so we do the same thing, though
   it is apparently not strictly necessary.
   
   PR:		187312
 
 Modified:
   stable/10/sys/dev/mfi/mfi_pci.c
   stable/10/sys/dev/mfi/mfi_tbolt.c
   stable/10/sys/dev/mfi/mfivar.h
 Directory Properties:
   stable/10/   (props changed)
 
 Modified: stable/10/sys/dev/mfi/mfi_pci.c
 ==============================================================================
 --- stable/10/sys/dev/mfi/mfi_pci.c	Mon Mar 10 01:34:38 2014	(r262966)
 +++ stable/10/sys/dev/mfi/mfi_pci.c	Mon Mar 10 02:31:50 2014	(r262967)
 @@ -131,7 +131,8 @@ struct mfi_ident {
  	{0x1000, 0x005b, 0x8086, 0x9265, MFI_FLAGS_SKINNY| MFI_FLAGS_TBOLT, "Intel (R) RAID Controller RS25DB080"},
  	{0x1000, 0x005b, 0x8086, 0x9285, MFI_FLAGS_SKINNY| MFI_FLAGS_TBOLT, "Intel (R) RAID Controller RS25NB008"},
  	{0x1000, 0x005b, 0xffff, 0xffff, MFI_FLAGS_SKINNY| MFI_FLAGS_TBOLT, "ThunderBolt"},
 -	{0x1000, 0x005d, 0xffff, 0xffff, MFI_FLAGS_SKINNY| MFI_FLAGS_TBOLT, "Invader"},
 +	{0x1000, 0x005d, 0xffff, 0xffff, MFI_FLAGS_SKINNY| MFI_FLAGS_TBOLT| MFI_FLAGS_INVADER, "Invader"},
 +	{0x1000, 0x005f, 0xffff, 0xffff, MFI_FLAGS_SKINNY| MFI_FLAGS_TBOLT| MFI_FLAGS_FURY, "Fury"},
  	{0x1000, 0x0060, 0x1028, 0xffff, MFI_FLAGS_1078,  "Dell PERC 6"},
  	{0x1000, 0x0060, 0xffff, 0xffff, MFI_FLAGS_1078,  "LSI MegaSAS 1078"},
  	{0x1000, 0x0071, 0xffff, 0xffff, MFI_FLAGS_SKINNY, "Drake Skinny"},
 
 Modified: stable/10/sys/dev/mfi/mfi_tbolt.c
 ==============================================================================
 --- stable/10/sys/dev/mfi/mfi_tbolt.c	Mon Mar 10 01:34:38 2014	(r262966)
 +++ stable/10/sys/dev/mfi/mfi_tbolt.c	Mon Mar 10 02:31:50 2014	(r262967)
 @@ -850,7 +850,8 @@ mfi_tbolt_build_ldio(struct mfi_softc *s
  	io_request = cmd->io_request;
  	io_request->RaidContext.TargetID = device_id;
  	io_request->RaidContext.Status = 0;
 -	io_request->RaidContext.exStatus =0;
 +	io_request->RaidContext.exStatus = 0;
 +	io_request->RaidContext.regLockFlags = 0;
  
  	start_lba_lo = mfi_cmd->cm_frame->io.lba_lo;
  	start_lba_hi = mfi_cmd->cm_frame->io.lba_hi;
 @@ -945,6 +946,7 @@ mfi_tbolt_make_sgl(struct mfi_softc *sc,
  	uint8_t i, sg_processed, sg_to_process;
  	uint8_t sge_count, sge_idx;
  	union mfi_sgl *os_sgl;
 +	pMpi25IeeeSgeChain64_t sgl_end;
  
  	/*
  	 * Return 0 if there is no data transfer
 @@ -968,6 +970,11 @@ mfi_tbolt_make_sgl(struct mfi_softc *sc,
  	else
  		sge_idx = sge_count;
  
 +	if (sc->mfi_flags & (MFI_FLAGS_INVADER | MFI_FLAGS_FURY)) {
 +		sgl_end = sgl_ptr + (sc->max_SGEs_in_main_message - 1);
 +		sgl_end->Flags = 0;
 +	}
 +
  	for (i = 0; i < sge_idx; i++) {
  		/*
  		 * For 32bit BSD we are getting 32 bit SGL's from OS
 @@ -981,7 +988,11 @@ mfi_tbolt_make_sgl(struct mfi_softc *sc,
  			sgl_ptr->Length = os_sgl->sg32[i].len;
  			sgl_ptr->Address = os_sgl->sg32[i].addr;
  		}
 -		sgl_ptr->Flags = 0;
 +		if (i == sge_count - 1 &&
 +		    (sc->mfi_flags & (MFI_FLAGS_INVADER | MFI_FLAGS_FURY)))
 +			sgl_ptr->Flags = MPI25_IEEE_SGE_FLAGS_END_OF_LIST;
 +		else
 +			sgl_ptr->Flags = 0;
  		sgl_ptr++;
  		cmd->io_request->ChainOffset = 0;
  	}
 @@ -996,8 +1007,11 @@ mfi_tbolt_make_sgl(struct mfi_softc *sc,
  		sg_chain = sgl_ptr;
  		/* Prepare chain element */
  		sg_chain->NextChainOffset = 0;
 -		sg_chain->Flags = (MPI2_IEEE_SGE_FLAGS_CHAIN_ELEMENT |
 -		    MPI2_IEEE_SGE_FLAGS_IOCPLBNTA_ADDR);
 +		if (sc->mfi_flags & (MFI_FLAGS_INVADER | MFI_FLAGS_FURY))
 +			sg_chain->Flags = MPI2_IEEE_SGE_FLAGS_CHAIN_ELEMENT;
 +		else
 +			sg_chain->Flags = MPI2_IEEE_SGE_FLAGS_CHAIN_ELEMENT |
 +			    MPI2_IEEE_SGE_FLAGS_IOCPLBNTA_ADDR;
  		sg_chain->Length =  (sizeof(MPI2_SGE_IO_UNION) *
  		    (sge_count - sg_processed));
  		sg_chain->Address = cmd->sg_frame_phys_addr;
 @@ -1010,7 +1024,13 @@ mfi_tbolt_make_sgl(struct mfi_softc *sc,
  				sgl_ptr->Length = os_sgl->sg32[i].len;
  				sgl_ptr->Address = os_sgl->sg32[i].addr;
  			}
 -			sgl_ptr->Flags = 0;
 +			if (i == sge_count - 1 &&
 +			    (sc->mfi_flags &
 +			    (MFI_FLAGS_INVADER | MFI_FLAGS_FURY)))
 +				sgl_ptr->Flags =
 +				    MPI25_IEEE_SGE_FLAGS_END_OF_LIST;
 +			else
 +				sgl_ptr->Flags = 0;
  			sgl_ptr++;
  		}
  	}
 
 Modified: stable/10/sys/dev/mfi/mfivar.h
 ==============================================================================
 --- stable/10/sys/dev/mfi/mfivar.h	Mon Mar 10 01:34:38 2014	(r262966)
 +++ stable/10/sys/dev/mfi/mfivar.h	Mon Mar 10 02:31:50 2014	(r262967)
 @@ -199,6 +199,8 @@ struct mfi_softc {
  #define MFI_FLAGS_GEN2		(1<<6)
  #define MFI_FLAGS_SKINNY	(1<<7)
  #define MFI_FLAGS_TBOLT		(1<<8)
 +#define MFI_FLAGS_INVADER	(1<<10)
 +#define MFI_FLAGS_FURY		(1<<11)
  	// Start: LSIP200113393
  	bus_dma_tag_t			verbuf_h_dmat;
  	bus_dmamap_t			verbuf_h_dmamap;
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: markj 
State-Changed-When: Mon Mar 10 03:05:02 UTC 2014 
State-Changed-Why:  
The change was committed to stable/10 as r262967. Thanks for the PR! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=187312 
>Unformatted:
