From nobody@FreeBSD.org  Mon Mar 10 11:39:39 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id 7FF76317
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 10 Mar 2014 11:39:39 +0000 (UTC)
Received: from cgiserv.freebsd.org (cgiserv.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mx1.freebsd.org (Postfix) with ESMTPS id 6E05CC54
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 10 Mar 2014 11:39:39 +0000 (UTC)
Received: from cgiserv.freebsd.org ([127.0.1.6])
	by cgiserv.freebsd.org (8.14.8/8.14.8) with ESMTP id s2ABddcr069148
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 10 Mar 2014 11:39:39 GMT
	(envelope-from nobody@cgiserv.freebsd.org)
Received: (from nobody@localhost)
	by cgiserv.freebsd.org (8.14.8/8.14.8/Submit) id s2ABddJS069144;
	Mon, 10 Mar 2014 11:39:39 GMT
	(envelope-from nobody)
Message-Id: <201403101139.s2ABddJS069144@cgiserv.freebsd.org>
Date: Mon, 10 Mar 2014 11:39:39 GMT
From: Gordon Yeu <kmd@kmd.twbbs.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ZFS Write Deadlock
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         187414
>Category:       kern
>Synopsis:       [zfs] ZFS Write Deadlock on 8.4
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-fs
>State:          suspended
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 10 11:40:00 UTC 2014
>Closed-Date:    
>Last-Modified:  Wed Apr 16 14:10:00 UTC 2014
>Originator:     Gordon Yeu
>Release:        8.4-STABLE
>Organization:
>Environment:
FreeBSD kmd.twbbs.org 8.4-STABLE FreeBSD 8.4-STABLE #6 r262357: Sun Feb 23 18:38:39 CST 2014     root@kmd.twbbs.org:/usr/obj/usr/src/sys/MERCURY  amd64
>Description:
Just did a periodical update to 8.4-STABLE r262357 on a self-build home
server. The only change I did after update is upgrade zpool version to
latest (version 28). My zpool is in mirror configuration, built from dual
Seagate 3TB hard drives (ST3000DM001). 


Moving large files in ~4GB size hangs the target directory, and Bonnie++
leads to deadlock as well. So I believe it was caused by large write
operations. 

Here are logs for both cases. 
mv: procstat -kk -a 
http://pastebin.com/WManZzbv 

bonnie++: procstat -kk -a 
http://pastebin.com/fe2bmbbY 

bonnie++: kgdb (thread apply all bt) 
http://pastebin.com/kdm5xQwM 
>How-To-Repeat:
run bonnie++ and set -d scratch-dir on ZFS.
>Fix:


>Release-Note:
>Audit-Trail:

From: Gordon Yeu <kmd@kmd.twbbs.org>
To: bug-followup@FreeBSD.org, kmd@kmd.twbbs.org
Cc:  
Subject: Re: kern/187414: ZFS Write Deadlock
Date: Tue, 18 Mar 2014 17:11:51 +0800

 --001a1135ed9639bcbf04f4ddeddd
 Content-Type: text/plain; charset=ISO-8859-1
 
 Clean install FreeBSD 10.0-STABLE r263205, everything works fine. Time to
 say goodbye to 8.4-STABLE.
 
 uname -a
 FreeBSD kmd.twbbs.org 10.0-STABLE FreeBSD 10.0-STABLE #0 r263205: Sat Mar
 15 23:38:52 CST 2014     root@kmd.twbbs.org:/usr/obj/usr/src/sys/MERCURY
  amd64
 
 --001a1135ed9639bcbf04f4ddeddd--
State-Changed-From-To: open->suspended 
State-Changed-By: linimon 
State-Changed-When: Wed Apr 16 01:43:20 UTC 2014 
State-Changed-Why:  
Clean up synopsis and assignment.  Setting to Suspended since submitter 
notes it no longer happens with 10.0, but leaving in GNATS to warn others. 


Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Apr 16 01:43:20 UTC 2014 
Responsible-Changed-Why:  

http://www.freebsd.org/cgi/query-pr.cgi?pr=187414 

From: John Baldwin <john@baldwin.cx>
To: bug-followup@freebsd.org,
 kmd@kmd.twbbs.org
Cc:  
Subject: Re: kern/187414: [zfs] ZFS Write Deadlock on 8.4
Date: Wed, 16 Apr 2014 09:39:26 -0400

 -- 
 John Baldwin

From: John Baldwin <jhb@freebsd.org>
To: bug-followup@freebsd.org,
 kmd@kmd.twbbs.org
Cc:  
Subject: Re: kern/187414: [zfs] ZFS Write Deadlock on 8.4
Date: Wed, 16 Apr 2014 09:40:31 -0400

 If the processes are hung in the "dmu_tx_delay" wait channel, then this might 
 have been fixed by:
 
 http://svnweb.freebsd.org/base?view=revision&revision=264505
 
 -- 
 John Baldwin
>Unformatted:
