From nobody@FreeBSD.org  Sun Jan 17 17:51:00 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CEC3D1065693
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 17 Jan 2010 17:50:59 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id B5D4C8FC14
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 17 Jan 2010 17:50:59 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o0HHoxIH034873
	for <freebsd-gnats-submit@FreeBSD.org>; Sun, 17 Jan 2010 17:50:59 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id o0HHoxaM034872;
	Sun, 17 Jan 2010 17:50:59 GMT
	(envelope-from nobody)
Message-Id: <201001171750.o0HHoxaM034872@www.freebsd.org>
Date: Sun, 17 Jan 2010 17:50:59 GMT
From: Miks <miks.mikelsons@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ZFS performance degratation over time
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         142914
>Category:       kern
>Synopsis:       [zfs] ZFS performance degradation over time
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-fs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Jan 17 18:00:07 UTC 2010
>Closed-Date:    Wed Jul 27 08:06:31 UTC 2011
>Last-Modified:  Wed Jul 27 08:06:31 UTC 2011
>Originator:     Miks
>Release:        Freebsd 8.0 Release
>Organization:
>Environment:
FreeBSD host1 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
After migrating servers to from UFS to ZFS, we got constant file system
performance degradation in couple time by week.

System load has not been changed since migration to ZFS, so only reason
of this problems can be ZFS itself.

We had this problems on Intel server systems with 8GB and more RAM.  We
also tried to disable prefetch, but problem still existed.  Even with
latest FreeBSD 8.0 Stable we had same problem.

One solution to this problem is - stop all httpd, php-cgi and mysql
processes, wait 1-2 minutes and then start it again.  On other servers
we just migrated back to UFS and problem was gone.

One strange thing we noticed is - when server have performance problem,
"zpool iostat" is showing like this
"
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zroot        113G  26.1G     53    392  2.80M  20.3M
"

after processes restart:
"
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
zroot       99.9G  39.1G      6     40   367K  1.12M

"

How can "used" value differ?
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-fs 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sun Jan 17 20:12:46 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=142914 

From: Ivan Voras <ivoras@fer.hr>
To: bug-followup@FreeBSD.org, miks.mikelsons@gmail.com
Cc:  
Subject: Re: kern/142914: [zfs] ZFS performance degradation over time
Date: Mon, 18 Jan 2010 10:52:04 +0100

 This is a multi-part message in MIME format.
 --------------020105090205050006040605
 Content-Type: text/plain; charset=UTF-8; format=flowed
 Content-Transfer-Encoding: 7bit
 
 RE: "used value differs": Have you checked with "du" the status "before" 
 and "after" to rule out that something is actually allocating a big file?
 
 There is a similar problem reported here:
 http://permalink.gmane.org/gmane.os.freebsd.stable/66780
 
 --------------020105090205050006040605
 Content-Type: text/x-vcard; charset=utf-8;
  name="ivoras.vcf"
 Content-Transfer-Encoding: base64
 Content-Disposition: attachment;
  filename="ivoras.vcf"
 
 YmVnaW46dmNhcmQNCmZuOkl2YW4gVm9yYXMNCm46Vm9yYXM7SXZhbg0Kb3JnOlVuaXZlcnNp
 dHkgb2YgWmFncmViO0ZhY3VsdHkgb2YgZWxlY3RyaWNhbCBlbmdpbmVlcmluZyBhbmQgY29t
 cHV0aW5nDQphZHI6OztVbnNrYSAzO1phZ3JlYjs7MTAwMDA7Q3JvYXRpYQ0KZW1haWw7aW50
 ZXJuZXQ6aXZvcmFzQGZlci5ocg0KdGl0bGU6SW50ZXJuZXQgc2VydmljZXMgYXJjaGl0ZWN0
 DQp0ZWw7d29yazorMzg1IDEgNjEyOSA2NjANCngtbW96aWxsYS1odG1sOkZBTFNFDQp2ZXJz
 aW9uOjIuMQ0KZW5kOnZjYXJkDQoNCg==
 --------------020105090205050006040605--

From: Martin Matuska <mm@FreeBSD.org>
To: bug-followup@FreeBSD.org, miks.mikelsons@gmail.com
Cc:  
Subject: Re: kern/142914: [zfs] ZFS performance degradation over time
Date: Mon, 18 Jul 2011 13:57:00 +0200

 Any news on this PR? Can we close it?
 Is the problem still present with ZFS v28 and latest 8-STABLE or 9-CURRENT?
 
 -- 
 Martin Matuska
 FreeBSD committer
 http://blog.vx.sk
 
State-Changed-From-To: open->closed 
State-Changed-By: mm 
State-Changed-When: Wed Jul 27 08:06:30 UTC 2011 
State-Changed-Why:  
Closed on submitter request. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=142914 
>Unformatted:
