From nobody@FreeBSD.org  Thu Apr 15 06:48:07 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id BD6F2106566C
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 15 Apr 2010 06:48:07 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id AB3B38FC19
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 15 Apr 2010 06:48:07 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o3F6m7Rn057709
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 15 Apr 2010 06:48:07 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id o3F6m71f057708;
	Thu, 15 Apr 2010 06:48:07 GMT
	(envelope-from nobody)
Message-Id: <201004150648.o3F6m71f057708@www.freebsd.org>
Date: Thu, 15 Apr 2010 06:48:07 GMT
From: Daniel Black <daniel.subs@internode.on.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [zfs] cannot offline two drives in a raidz2 configuration (8.0-stable)
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         145712
>Category:       kern
>Synopsis:       [zfs] cannot offline two drives in a raidz2 configuration (8.0-stable)
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-fs
>State:          suspended
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Apr 15 06:50:01 UTC 2010
>Closed-Date:    
>Last-Modified:  Thu May 13 22:29:18 UTC 2010
>Originator:     Daniel Black
>Release:        8.0
>Organization:
OVEE
>Environment:
FreeBSD brm00.smartcars.in.nicta.com.au 8.0-STABLE FreeBSD 8.0-STABLE #0: Fri Apr 16 01:53:45 EST 2010     root@brm00.smartcars.in.nicta.com.au:/usr/obj/usr/src/sys/BRM  amd64

The stable was based on cvsup as of a few hours ago.
>Description:
raidz2 should support two disks being offline.

The kernel implementation does not support this.

Noted but not problem reported here: http://lists.freebsd.org/pipermail/freebsd-stable/2010-February/054862.html
>How-To-Repeat:
I have two raidz2 virtual devices in the tank pool.

# zpool status -v
  pool: tank
 state: ONLINE
status: The pool is formatted using an older on-disk format.  The pool can
	still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
	pool will no longer be accessible on older software versions.
 scrub: scrub in progress for 0h11m, 3.30% done, 5h40m to go
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  raidz2    ONLINE       0     0     0
	    ada10   ONLINE       0     0     0
	    ada11   ONLINE       0     0     0
	    ada12   ONLINE       0     0     0
	    ada13   ONLINE       0     0     0
	    ada14   ONLINE       0     0     0
	  raidz2    ONLINE       0     0     0
	    ada5    ONLINE       0     0     0
	    ada6    ONLINE       0     0     0
	    ada7    ONLINE       0     0     0
	    ada8    ONLINE       0     0     0
	    ada9    ONLINE       0     0     0
	spares
	  ada0      AVAIL   
	  ada1      AVAIL   
	  ada2      AVAIL   
	  ada3      AVAIL   
	  ada4      AVAIL   

# zpool offline tank ada10
(works)
# zpool offline tank ada11
cannot offline ada11: no valid replicas

# zpool status -v
  pool: tank
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: scrub in progress for 0h21m, 5.78% done, 5h51m to go
config:

	NAME        STATE     READ WRITE CKSUM
	tank        DEGRADED     0     0     0
	  raidz2    DEGRADED     0     0     0
	    ada10   OFFLINE      0 10.6K     0
	    ada11   ONLINE       0     0     0
	    ada12   ONLINE       0     0     0
	    ada13   ONLINE       0     0     0
	    ada14   ONLINE       0     0     0
	  raidz2    ONLINE       0     0     0
	    ada5    ONLINE       0     0     0
	    ada6    ONLINE       0     0     0
	    ada7    ONLINE       0     0     0
	    ada8    ONLINE       0     0     0
	    ada9    ONLINE       0     0     0
	spares
	  ada0      AVAIL   
	  ada1      AVAIL   
	  ada2      AVAIL   
	  ada3      AVAIL   
	  ada4      AVAIL   

# truss zpool offline tank ada11
..
ioctl(2,0xcc285a05 { IORW 0x5a('Z'), 5, 3112 },0xffff9ec0) = 0 (0x0)
ioctl(2,0xcc285a0d { IORW 0x5a('Z'), 13, 3112 },0xffff9b00) ERR#16 'Device busy'
write(2,"cannot offline ada11: no valid r"...,40) ERR#19 'Operation not supported by device'


I don't believe this is affected by the current scrub status.
>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Apr 15 08:16:31 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=145712 
State-Changed-From-To: open->suspended 
State-Changed-By: pjd 
State-Changed-When: czw 13 maj 2010 22:26:48 UTC 
State-Changed-Why:  
This problem is fixed in recent ZFS, but the fix is too intrusive to 
back-port it. It has to wait for new ZFS import. 

BTW. This issue also exists for N-way mirrors where N > 2. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=145712 
>Unformatted:
