From nobody@FreeBSD.org  Tue May 27 11:57:43 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id B0A058C1
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 27 May 2014 11:57:43 +0000 (UTC)
Received: from cgiserv.freebsd.org (cgiserv.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 9DB052067
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 27 May 2014 11:57:43 +0000 (UTC)
Received: from cgiserv.freebsd.org ([127.0.1.6])
	by cgiserv.freebsd.org (8.14.8/8.14.8) with ESMTP id s4RBvh9d082113
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 27 May 2014 11:57:43 GMT
	(envelope-from nobody@cgiserv.freebsd.org)
Received: (from nobody@localhost)
	by cgiserv.freebsd.org (8.14.8/8.14.8/Submit) id s4RBvhx7082112;
	Tue, 27 May 2014 11:57:43 GMT
	(envelope-from nobody)
Message-Id: <201405271157.s4RBvhx7082112@cgiserv.freebsd.org>
Date: Tue, 27 May 2014 11:57:43 GMT
From: Mykola Zubach <zuborg@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: vmdaemon kernel thread consumes 100% cpu
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         190300
>Category:       kern
>Synopsis:       vmdaemon kernel thread consumes 100% cpu
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 27 12:00:00 UTC 2014
>Closed-Date:    
>Last-Modified:  Tue May 27 13:50:00 UTC 2014
>Originator:     Mykola Zubach
>Release:        10.0-RELEASE
>Organization:
advancedhosters.com
>Environment:
FreeBSD DS4333 10.0-RELEASE-p3 FreeBSD 10.0-RELEASE-p3 #0 r266540: Thu May 22 12:28:51 UTC 2014     root@DS4333:/usr/obj/usr/src/sys/Z-AMD64  amd64
>Description:
10.0 release have a bug which cause `vmdaemon` kernel thread to consume all available cpu.

Server run postgresql-9.3 (which uses mmap() actively) and ~30 perl processes which also use mmap() for its data files.


last pid: 97992;  load averages:  3.34,  4.16,  4.39                                                                                                                  up 4+23:02:49  11:50:38
144 processes: 6 running, 123 sleeping, 1 waiting, 14 lock
CPU:  8.0% user,  0.0% nice,  4.8% system,  0.3% interrupt, 86.9% idle
Mem: 14G Active, 42G Inact, 4289M Wired, 122M Cache, 1668M Buf, 1240M Free
Swap: 4096M Total, 4096M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
   10 root         24 155 ki31     0K   384K CPU23  23 2845.2 2065.48% idle
   16 root          1 -16    -     0K    16K RUN     0  23:23  98.73% vmdaemon
76669 cdn           1  31    0  7270M  2634M accept  4  13:04  18.26% perl5.16.3
76674 cdn           1  30    0  7270M  2628M accept  7  12:38  17.14% perl5.16.3
76665 cdn           1  29    0  7270M  2610M accept  5  12:00  15.87% perl5.16.3
76672 cdn           1  28    0  7270M  2649M accept  2  14:34  15.58% perl5.16.3
76668 cdn           1  28    0  7270M  2621M accept 10  12:02  15.43% perl5.16.3
76673 cdn           1  29    0  7270M  2632M accept 18  12:58  14.99% perl5.16.3
76667 cdn           1  27    0  7271M  2611M CPU14  14  11:55  14.75% perl5.16.3
76676 cdn           1  27    0  7274M  2648M CPU7    7  13:41  14.65% perl5.16.3
76670 cdn           1  28    0  7270M  2621M accept 12  12:12  14.55% perl5.16.3
76671 cdn           1  27    0  7270M  2611M accept 22  12:02  13.92% perl5.16.3
76675 cdn           1  28    0  7270M  2600M accept  2  11:57  13.62% perl5.16.3
76666 cdn           1  25    0  7270M  2619M select 13  12:25   9.81% perl5.16.3
76695 cdn           1  26    0  7294M  2504M select 19   4:06   8.01% perl5.16.3
76702 cdn           1  23    0  7294M  2514M select  5   4:15   5.57% perl5.16.3
76701 cdn           1  25    0  7294M  2511M select 21   4:10   5.42% perl5.16.3
76699 cdn           1  24    0  7294M  2488M select 20   4:01   5.42% perl5.16.3
76694 cdn           1  25    0  7294M  2490M select  7   3:53   5.22% perl5.16.3
76691 cdn           1  23    0  7295M  2493M select 12   3:49   5.13% perl5.16.3
   15 root          1 -16    -     0K    16K psleep  6  10:34   4.54% pagedaemon
76693 cdn           1  23    0  7295M  2486M select 17   3:52   4.39% perl5.16.3
76700 cdn           1  22    0  7294M  2500M select  9   3:53   3.86% perl5.16.3
76698 cdn           1  22    0  7294M  2486M select 20   3:54   3.52% perl5.16.3
76692 cdn           1  22    0  7290M  2482M select 22   3:44   3.03% perl5.16.3
76697 cdn           1  22    0  7290M  2466M select 10   3:43   2.78% perl5.16.3
76715 pgsql         1  22    0 21253M  5046M *vm ob 14   1:38   2.54% postgres
76696 cdn           1  21    0  7294M  2470M select  6   3:44   2.44% perl5.16.3
76687 pgsql         1  25    0 21253M   140M sbwait 20   0:41   1.90% postgres
76683 pgsql         1  24    0 21253M   139M sbwait  6   0:44   1.86% postgres
42720 pgsql         1  23    0 21253M 13997M *vm ob 14  21:26   1.32% postgres
76710 pgsql         1  23    0 21253M  4971M *vm ob  8   1:37   1.27% postgres
76706 pgsql         1  23    0 21253M  5005M *vm ob 14   1:38   0.83% postgres
76704 pgsql         1  23    0 21253M  5011M *vm ob 21   1:38   0.83% postgres
76713 pgsql         1  23    0 21253M  5024M *vm ob  2   1:37   0.83% postgres
76711 pgsql         1  23    0 21253M  4964M *vm ob 23   1:36   0.83% postgres
76712 pgsql         1  24    0 21253M  4994M *vm ob 23   1:37   0.63% postgres
76707 pgsql         1  22    0 21253M  5023M *vm ob  3   1:38   0.59% postgres
76708 pgsql         1  22    0 21253M  5024M *vm ob  8   1:37   0.59% postgres

>How-To-Repeat:
I don't know exactly, may be run postgresql-9.3 under some load.
>Fix:


>Release-Note:
>Audit-Trail:

From: Mykola Zubach <zuborg@gmail.com>
To: bug-followup@freebsd.org
Cc:  
Subject: Re: kern/190300: vmdaemon kernel thread consumes 100% cpu
Date: Tue, 27 May 2014 16:45:05 +0300

 --90e6ba6e8d0647115004fa61e7e0
 Content-Type: text/plain; charset=UTF-8
 
 FreeBSD 9.x is not affected
 
 `swapoff -a` didn't help
 
 When I've reduced load on server -' vmdaemon' continues to use 100% cpu,
 but stop doing it when I've restarted perl worker processes.
 
 --90e6ba6e8d0647115004fa61e7e0
 Content-Type: text/html; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable
 
 <div dir=3D"ltr"><div>FreeBSD 9.x is not affected</div><div><br></div><div>=
 `swapoff -a` didn&#39;t help<br></div><div><br></div><div>When I&#39;ve red=
 uced load on server -&#39; vmdaemon&#39; continues to use 100% cpu, but sto=
 p doing it when I&#39;ve restarted perl worker processes.</div>
 </div>
 
 --90e6ba6e8d0647115004fa61e7e0--
>Unformatted:
