From nobody@FreeBSD.org  Tue May 24 14:39:49 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 63E311065678
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 24 May 2011 14:39:49 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 3AD338FC1A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 24 May 2011 14:39:49 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p4OEdn5B095127
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 24 May 2011 14:39:49 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p4OEdnq9095126;
	Tue, 24 May 2011 14:39:49 GMT
	(envelope-from nobody)
Message-Id: <201105241439.p4OEdnq9095126@red.freebsd.org>
Date: Tue, 24 May 2011 14:39:49 GMT
From: Jonathan <jonathan@kc8onw.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: mfiutil/mfi does not allow adding a previously configured drive to be added to a new array
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         157293
>Category:       kern
>Synopsis:       [mfi] mfiutil/mfi does not allow adding a previously configured drive to be added to a new array
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    smh
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 24 14:40:10 UTC 2011
>Closed-Date:    Sat Mar 02 01:26:56 UTC 2013
>Last-Modified:  Sat Mar 02 01:26:56 UTC 2013
>Originator:     Jonathan
>Release:        8.2-STABLE
>Organization:
>Environment:
FreeBSD SERENITY.kc8onw.net 8.2-STABLE FreeBSD 8.2-STABLE #2 r219138: Tue Mar  1 13:42:29 EST 2011     root@SERENITY.kc8onw.net:/usr/obj/usr/src/sys/SERENITY  amd64
>Description:
My server lost power and the RAID controller did not see the 2TB drive
when the system came back up.  The tech at the datacenter told the
controller to continue booting despite the drive being "missing" and
then removed and re-inserted the drive at my request.  Now that the
system has booted the RAID controller sees the drives as shown below.

SERENITY# mfiutil show drives
mfi0 Physical Drives:
(  419G) ONLINE            <SEAGATE ST3450802SS 0002 serial=3SR00N6E> SAS enclosure 1, slot 0
(  419G) ONLINE            <SEAGATE ST3450802SS 0002 serial=3SR00JN1> SAS enclosure 1, slot 1
(  419G) ONLINE            <SEAGATE ST3450802SS 0002 serial=3SR00DCQ> SAS enclosure 1, slot 2
( 1863G) UNCONFIGURED GOOD <WDC WD2002FYPS-0 1G01 serial=WD-WCAVY5131503> SATA enclosure 1, slot 3

If I try to reconfigure the drive as a JBOD or RAID0 I get the error
SERENITY# mfiutil create jbod E01:S03
mfiutil: Command failed: Wrong firmware or drive state
mfiutil: Failed to add volume: Input/output error
SERENITY# mfiutil create raid0 E01:S03
mfiutil: Command failed: Wrong firmware or drive state
mfiutil: Failed to add volume: Input/output error

Is there anyway to do this with mfiutil or will I have to go onsite (a
3 hour drive one way) and try to reconfigure the drive via the controller
BIOS?

I also found this report from 9 months ago that had the same problem but
I did not see a resolution
http://lists.freebsd.org/pipermail/freebsd-scsi/2010-September/004526.html
>How-To-Repeat:
Temporarily lose a drive on a controller that uses the mfi driver and
after telling the RAID control to continue booting anyway try to
re-create the array/jbod device with mfiutil.
>Fix:
>Release-Note:
>Audit-Trail:

From: Garrett Cooper <yanegomi@gmail.com>
To: bug-followup@FreeBSD.org, jonathan@kc8onw.net
Cc:  
Subject: Re: kern/157293: [mfi] mfiutil/mfi does not allow adding a previously
 configured drive to be added to a new array
Date: Fri, 9 Sep 2011 01:18:05 -0700 (PDT)

 I wish I had seen this issue earlier so I could have sent a suggestion 
 (in all fairness this should have been a post to an email list, like 
 freebsd-scsi, freebsd-stable, or even freebsd-questions.
 
 If you have a degraded stripe-only RAID, then you're basically SOL. That 
 aside.. if you didn't use a straight RAID-0, you could have readded the 
 new drive as a hot spare like so:
 
  	mfiutil add <drive-number> <volume>
 
 and the controller would have added the drive to your RAID.
 
 That's all you need to do when rebuilding a degraded, recoverable RAID via 
 mfiutil.
 
 HTH,
 -Garrett

From: Jonathan <jonathan@kc8onw.net>
To: Garrett Cooper <yanegomi@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/157293: [mfi] mfiutil/mfi does not allow adding a previously
 configured drive to be added to a new array
Date: Fri, 09 Sep 2011 10:20:26 -0400

 On 9/9/2011 4:18 AM, Garrett Cooper wrote:
 > I wish I had seen this issue earlier so I could have sent a suggestion
 > (in all fairness this should have been a post to an email list, like
 > freebsd-scsi, freebsd-stable, or even freebsd-questions.
 
 Noted.
 
 > If you have a degraded stripe-only RAID, then you're basically SOL. That
 > aside.. if you didn't use a straight RAID-0, you could have readded the
 > new drive as a hot spare like so:
  > mfiutil add <drive-number> <volume>
  > and the controller would have added the drive to your RAID.
 
 The drive in question was the only member of it's RAID stripe because I 
 could not get the controller to do pass through on it when I first set 
 it up.  For some reason the drive was not properly recognized after a 
 power failure and in the process of trying to walk the data center 
 through bringing in back over the phone it was somehow removed from the 
 controllers stored configuration.  The drive itself still had the LSI 
 metadata on it somewhere that made the controller think it was still in 
 use elsewhere so it refused to let me create a new RAID or JBOD device 
 with it.  Would the "add" command have still worked in that case, it 
 seems unlikely?  The original drive turned out to be fine.  I got it 
 from the datacenter, put it in my home server, and imported the ZFS pool 
 without a problem and with all of it's data intact and it's still there now.
 
 It seems like there needs to be a way to either force a drive into an 
 mfi RAID or to somehow wipe the controller metadata without having to dd 
 /dev/zero to the drive.

From: "Steven Hartland" <killing@multiplay.co.uk>
To: <bug-followup@freebsd.org>,
	"Jonathan" <jonathan@kc8onw.net>
Cc:  
Subject: Re: kern/157293: [mfi] mfiutil/mfi does not allow adding a previously configured drive to be added to a new array
Date: Wed, 26 Sep 2012 13:59:52 +0100

 I believe the cause of your issue is that the disks where
 marked as "foreign".
 
 This doesn't mean they are bad but the controller won't let
 you import, add etc as it thinks they are already a part of
 another array, so its just being safe.
 
 I've just finished off some updates to mfiutil which will
 both detect this and allow you to "fix" them.
 
 The PR for these fixes includes the patch:
 http://www.freebsd.org/cgi/query-pr.cgi?pr=172091
 
 ================================================
 This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 
 
 In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
 or return the E.mail to postmaster@multiplay.co.uk.
 
Responsible-Changed-From-To: freebsd-bugs->smh 
Responsible-Changed-By: smh 
Responsible-Changed-When: Thu Dec 13 22:43:49 UTC 2012 
Responsible-Changed-Why:  
Could you confirm or deny if your disks where marked as foreign or not? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=157293 
State-Changed-From-To: open->closed 
State-Changed-By: smh 
State-Changed-When: Sat Mar 2 01:26:55 UTC 2013 
State-Changed-Why:  
Feedback timeout 

http://www.freebsd.org/cgi/query-pr.cgi?pr=157293 
>Unformatted:
