From nobody@FreeBSD.org  Thu Jan 26 18:48:30 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 22BF7106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 26 Jan 2012 18:48:30 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id EBC3C8FC15
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 26 Jan 2012 18:48:29 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q0QImTpK087048
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 26 Jan 2012 18:48:29 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id q0QImTYE087047;
	Thu, 26 Jan 2012 18:48:29 GMT
	(envelope-from nobody)
Message-Id: <201201261848.q0QImTYE087047@red.freebsd.org>
Date: Thu, 26 Jan 2012 18:48:29 GMT
From: Eugen Konkov <kes-kes@yandex.ru>
To: freebsd-gnats-submit@FreeBSD.org
Subject: 'kill' can not kill process despite on -KILL 
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         164526
>Category:       bin
>Synopsis:       kill(1) can not kill process despite on -KILL
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jan 26 18:50:06 UTC 2012
>Closed-Date:    
>Last-Modified:  Thu Feb  2 21:30:11 UTC 2012
>Originator:     Eugen Konkov
>Release:        9.0-CURRENT
>Organization:
ISP FreeLine
>Environment:
# uname -a
FreeBSD 9.0-CURRENT FreeBSD 9.0-CURRENT #4: Fri Jun 10 01:30:12 UTC 2011     :/usr/obj/usr/src/sys/PAE_KES  i386

>Description:
man kill
.....
     Some of the more commonly used signals:
.....
     9       KILL (non-catchable, non-ignorable kill)

Standart tool do not do its job. It can not stop/kill processes.


 >> # ps ax|grep rad
 >> 45471  ??  T<Ls   263:35.44 /usr/local/sbin/radiusd
 >> 26473   1  S+       0:00.00 grep rad
 >> # date
 >> Fri Jan 20 23:20:28 UTC 2012
 >> # kill -KILL 45471
 >> # date
 >> Fri Jan 20 23:20:41 UTC 2012
 >> # kill -KILL 45471
 >> # date
 >> Fri Jan 20 23:20:54 UTC 2012
 >> # kill -KILL 45471
 >> 
 >> 
 >> top
 >>     9 root        16    -     0K     8K syncer  2   7:12  0.00% syncer
 >> 45471 freeradius  20  -20   311M   283M STOP    0   3:38  0.00% {radiusd}
 >> 49114 root        21    0 10460K  4240K select  0   2:43  0.00% zebra
 
 # date
 Thu Jan 26 20:45:17 UTC 2012
 # ps ax | grep rad
 45471  ??  T<Ls   263:35.44 /usr/local/sbin/radiusd
 22879   1  R+       0:00.00 grep rad
 
 
 #top
 last pid: 23814;  load averages:  4.83,  5.42,  5.37                                                                               up 15+04:54:35  20:46:28
 242 processes: 10 running, 211 sleeping, 3 stopped, 17 waiting, 1 lock
 CPU 0:  3.4% user,  0.0% nice, 20.7% system, 55.2% interrupt, 20.7% idle
 CPU 1: 12.1% user,  0.0% nice, 10.3% system, 58.6% interrupt, 19.0% idle
 CPU 2:  3.4% user,  0.0% nice,  8.6% system, 62.1% interrupt, 25.9% idle
 CPU 3:  1.7% user,  0.0% nice,  5.2% system, 75.9% interrupt, 17.2% idle
 Mem: 842M Active, 2509M Inact, 335M Wired, 151M Cache, 112M Buf, 61M Free
 Swap: 4096M Total, 103M Used, 3993M Free, 2% Inuse
 
 ........
 45471 freeradius  20  -20   311M   197M STOP    3   1:52  0.00% {radiusd}
 
 
 if radiusd is locked, how to kill locked process???
 if 'kill -KILL' is non-catchable and  non-ignorable kill, 
 allow 'kill -KILL' to kill locked processes.
 
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:

From: =?windows-1251?B?yu7t/Oru4iDF4uPl7ejp?= <kes-kes@yandex.ru>
To: bug-followup@FreeBSD.org, kes-kes@yandex.ru
Cc:  
Subject: Re: bin/164526: kill(1) can not kill process despite on -KILL
Date: Fri, 27 Jan 2012 08:03:13 +0200

 I try to reboot server remotely.
 
 #reboot
 
 and it hangs up. host is pinging but I can not connect to it
 
 

From: Jilles Tjoelker <jilles@stack.nl>
To: bug-followup@FreeBSD.org, kes-kes@yandex.ru
Cc:  
Subject: Re: bin/164526: kill(1) can not kill process despite on -KILL
Date: Sat, 28 Jan 2012 19:24:07 +0100

 > [stuck process cannot be killed, system hangs when reboot is
 > attempted]
 
 A signal cannot forcibly kill a process that is stuck in the kernel.
 Allowing this would put the integrity of the kernel data structures at
 risk and likely cause hangs, data corruption or panics later on.
 
 If a process is stuck in the kernel for a long time, this can be things
 like broken hardware, a non-responsive NFS server or a bug.
 
 A state 'T' (stopped) probably means the process is multi-threaded and
 is trying to suspend but one or more threads will not cooperate
 (non-interruptible sleep or running in the kernel).
 
 Useful commands to obtain more information (supposing pid is 45471):
 
 ps Hl45471
 procstat -k 45471
 
 Of course, this does not help if you already rebooted.
 
 -- 
 Jilles Tjoelker

From: =?windows-1251?B?yu7t/Oru4iDF4uPl7ejp?= <kes-kes@yandex.ru>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@FreeBSD.org
Subject: Re: bin/164526: kill(1) can not kill process despite on -KILL
Date: Sun, 29 Jan 2012 00:07:03 +0200

 , Jilles.
 
   28  2012 ., 20:24:07:
 
 >> [stuck process cannot be killed, system hangs when reboot is
 >> attempted]
 
 JT> A signal cannot forcibly kill a process that is stuck in the kernel.
 JT> Allowing this would put the integrity of the kernel data structures at
 JT> risk and likely cause hangs, data corruption or panics later on.
 
 JT> If a process is stuck in the kernel for a long time, this can be things
 JT> like broken hardware, a non-responsive NFS server or a bug.
 
 JT> A state 'T' (stopped) probably means the process is multi-threaded and
 JT> is trying to suspend but one or more threads will not cooperate
 JT> (non-interruptible sleep or running in the kernel).
 
 JT> Useful commands to obtain more information (supposing pid is 45471):
 
 JT> ps Hl45471
 JT> procstat -k 45471
 
 JT> Of course, this does not help if you already rebooted.
 
 that was happen when: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/164145
 kern/164145: [netisr] when one of netisr threads take 100% system is going to work unstable
 
 at this moment I try to restart 'radiusd' and it was STOPped forever
 even when I try to reboot (((( whole system is stopped
 so I need to travel to server to turn it OFF.
 
 may be in this state just 'jmp BOOTSYSTEM_FROM_BEGINNING'
  (sorry, do no remember address to start BIOS POST)
 and do not allow to hangs forever.
 
 
 -- 
  ,
                            mailto:kes-kes@yandex.ru
 

From: =?windows-1251?B?yu7t/Oru4iDF4uPl7ejp?= <kes-kes@yandex.ru>
To: Jilles Tjoelker <jilles@stack.nl>
Cc: bug-followup@FreeBSD.org, freeradius-users@lists.freeradius.org, 
	<firebird-devel@lists.sourceforge.net>
Subject: Re[2]: bin/164526: kill(1) can not kill process despite on -KILL
Date: Thu, 2 Feb 2012 00:16:39 +0200

 , Jilles.
 
   28  2012 ., 20:24:07:
 
 >> [stuck process cannot be killed, system hangs when reboot is
 >> attempted]
 
 JT> A signal cannot forcibly kill a process that is stuck in the kernel.
 JT> Allowing this would put the integrity of the kernel data structures at
 JT> risk and likely cause hangs, data corruption or panics later on.
 
 JT> If a process is stuck in the kernel for a long time, this can be things
 JT> like broken hardware, a non-responsive NFS server or a bug.
 
 JT> A state 'T' (stopped) probably means the process is multi-threaded and
 JT> is trying to suspend but one or more threads will not cooperate
 JT> (non-interruptible sleep or running in the kernel).
 
 JT> Useful commands to obtain more information (supposing pid is 45471):
 
 JT> ps Hl45471
 JT> procstat -k 45471
 
 JT> Of course, this does not help if you already rebooted.
 
 repeated again:
 bug is repeateable:
 1. radiusd + mod_perl + example.pl(it is connects to FireBird) +
 FireBIrd
 2. restart firebird
 3. try to restart radiusd
 4. process in fall into STOP state
 
 # ps awx | grep radi
  9438  ??  TLs     5:10.12 /usr/local/sbin/radiusd
 27603   2  S+      0:00.00 grep radi
 # procstat -k 9438
   PID    TID COMM             TDNAME           KSTACK
  9438 100080 radiusd          -                mi_switch sleepq_switch sleepq_wait _sx_xlock_hard _sx_xlock _vm_map_lock_upgrade vm_map_lookup vm_fault_hold vm_fault trap_pfault trap calltrap
  9438 100195 radiusd          -                mi_switch sleepq_switch sleepq_wait __lockmgr_args ffs_lock VOP_LOCK1_APV _vn_lock vm_object_deallocate unlock_and_deallocate vm_fault_hold vm_fault trap_pfault trap calltrap
  9438 101144 radiusd          -                mi_switch thread_suspend_switch thread_single exit1 sigexit postsig ast doreti_ast
 # ps wHl9438
   UID   PID  PPID CPU PRI NI    VSZ    RSS MWCHAN STAT  TT     TIME COMMAND
   133  9438     1   0  20  0 351124 322000 user m TLs   ??  0:03.65 /usr/local/sbin/radiusd
   133  9438     1   0  20  0 351124 322000 ufs    TLs   ??  0:00.00 /usr/local/sbin/radiusd
   133  9438     1   0  20  0 351124 322000 -      TLs   ??  0:05.28 /usr/local/sbin/radiusd
 
 #top
 last pid: 28497;  load averages:  0.56,  2.34,  9.37                                                                    up 0+10:23:14  00:12:5
 162 processes: 1 running, 158 sleeping, 3 stopped
 CPU:  1.9% user,  0.0% nice,  1.9% system,  5.3% interrupt, 90.8% idle
 Mem: 525M Active, 1259M Inact, 182M Wired, 41M Cache, 112M Buf, 1890M Free
 Swap: 4096M Total, 4096M Free
 
   PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
  6893 root          1  26    0 15392K  5580K select  0  21:17  6.10% snmpd
 75797 bind          7  20    0   100M 77280K kqread  2   4:27  0.00% named
  5553 root          7  20    0 53544K 39832K select  1   0:19  0.00% mpd5
 77411 dhcpd         1  20    0 15032K  5360K select  3   0:18  0.00% dhcpd
  3605 root          1  20    0 10460K  4004K select  3   0:11  0.00% zebra
  5316 root          1  20    0  9616K  1244K select  1   0:10  0.00% syslogd
  9438 freeradius    3  20    0   343M   314M STOP    0   0:09  0.00% radiusd
 80843 mysql        26  20    0   402M   333M sbwait  0   0:05  0.00% mysqld
  3611 root          1  20    0 14660K  5348K select  2   0:05  0.00% bgpd
 80396 www           1  20    0 37908K 22876K lockf   1   0:01  0.00% httpd
 26278 root          1  20    0 33812K 15608K select  2   0:01  0.00% httpd
 10559 www           1  20    0 42004K 26768K lockf   1   0:01  0.00% httpd
 
 if I can supply another usefull debug info, answer as fast as you can, I can
 not wait too long. Thank you.
 
 
 -- 
  ,
                            mailto:kes-kes@yandex.ru
 

From: Alan Buxey <A.L.M.Buxey@lboro.ac.uk>
To: "kes-kes@yandex.ru" <kes-kes@yandex.ru>, "jilles@stack.nl"
	<jilles@stack.nl>
Cc: "firebird-devel@lists.sourceforge.net"
	<firebird-devel@lists.sourceforge.net>, "bug-followup@freebsd.org"
	<bug-followup@freebsd.org>, "freeradius-users@lists.freeradius.org"
	<freeradius-users@lists.freeradius.org>
Subject: Re: Re[2]: bin/164526: kill(1) can not kill process despite on -KILL
Date: Wed, 1 Feb 2012 22:24:55 +0000

 --_000_D8EC1E741BEA427983AEAD05C1996BD4lboroacuk_
 Content-Type: text/plain; charset="utf-8"
 Content-Transfer-Encoding: base64
 
 a2lsbCAtOSAgZG9lc24ndCB3YW50IHRvIHBsYXkgZWl0aGVyPw0KDQpSZWFkIHRoZSBkb2NzL2Rl
 YnVnZ2luZyBmaWxlLiBDb21waWxlIEZSIHdpdGggZGVidWdnaW5nL2RldmVsb3BlciBzdHVmZiBh
 bmQgcnVuIGl0IHVuZGVyIGdkYm0gY29udHJvbCBhbmQgZG8geW91ciBraWxsaW5nIG9mIGZpcmVi
 aXJkIGFnYWluLg0KDQpJdCdzIGxpa2VseSB0byBiZSB0aGUgUEVSTCBpbnRlZ3JhdGlvbiBhcyB0
 aGF0cyB3aGF0IHdpbGwgZ2V0IGRvbmUgaW4gd2hlbiBmaXJlYmlyZCBpcyByZXN0YXJ0ZWQuIEFy
 ZSB5b3UgcGVybCB3aXRoIHRocmVhZGluZz8NCg0KTm90IHN1cmUgd2h5IHlvdXIgZ29pbmcgdGhy
 b3VnaCBob29wcyBmb3IgZmlyZWJpcmQgYWNjZXNzLCBJJ20gc3VyZSB0aGVyZSdzIGEgbmF0aXZl
 IG1vZHVsZS4uLi4NCg0KYWxhbg0KDQoNCg==
 
 --_000_D8EC1E741BEA427983AEAD05C1996BD4lboroacuk_
 Content-Type: text/html; charset="utf-8"
 Content-Transfer-Encoding: base64
 
 DQpraWxsIC05ICZuYnNwO2RvZXNuJiMzOTt0IHdhbnQgdG8gcGxheSBlaXRoZXI/PGJyPjxicj5S
 ZWFkIHRoZSBkb2NzL2RlYnVnZ2luZyBmaWxlLiBDb21waWxlIEZSIHdpdGggZGVidWdnaW5nL2Rl
 dmVsb3BlciBzdHVmZiBhbmQgcnVuIGl0IHVuZGVyIGdkYm0gY29udHJvbCBhbmQgZG8geW91ciBr
 aWxsaW5nIG9mIGZpcmViaXJkIGFnYWluLjxicj48YnI+SXQmIzM5O3MgbGlrZWx5IHRvIGJlIHRo
 ZSBQRVJMIGludGVncmF0aW9uIGFzIHRoYXRzIHdoYXQgd2lsbCBnZXQgZG9uZSBpbiB3aGVuIGZp
 cmViaXJkIGlzIHJlc3RhcnRlZC4gQXJlIHlvdSBwZXJsIHdpdGggdGhyZWFkaW5nPzxicj48YnI+
 Tm90IHN1cmUgd2h5IHlvdXIgZ29pbmcgdGhyb3VnaCBob29wcyBmb3IgZmlyZWJpcmQgYWNjZXNz
 LCBJJiMzOTttIHN1cmUgdGhlcmUmIzM5O3MgYSBuYXRpdmUgbW9kdWxlLi4uLjxicj48YnI+YWxh
 bjxicj48YnI+PGJyPg0K
 
 --_000_D8EC1E741BEA427983AEAD05C1996BD4lboroacuk_--

From: =?utf-8?B?0JrQvtC90YzQutC+0LIg0JXQstCz0LXQvdC40Lk=?= <kes-kes@yandex.ru>
To: Alan Buxey <A.L.M.Buxey@lboro.ac.uk>
Cc: "jilles@stack.nl" <jilles@stack.nl>, 
	"firebird-devel@lists.sourceforge.net" <firebird-devel@lists.sourceforge.net>, 
	"bug-followup@freebsd.org" <bug-followup@freebsd.org>, 
	"freeradius-users@lists.freeradius.org" <freeradius-users@lists.freeradius.org>
Subject: Re[4]: bin/164526: kill(1) can not kill process despite on -KILL
Date: Thu, 2 Feb 2012 01:04:44 +0200

 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html><head><title>Re[4]: bin/164526: kill(1) can not kill process despite on -KILL</title>
 <META http-equiv=Content-Type content="text/html; charset=utf-8">
 <meta http-equiv="Content-Style-Type" content="text/css">
 <style type="text/css"><!--
 body {
   margin: 5px 5px 5px 5px;
   background-color: #ffffff;
 }
 </head>
 <body>
 
 <p>Здравствуйте, Alan.</p>
 <p><br></p>
 <p>Вы писали 2 февраля 2012 г., 0:24:55:</p>
 <p><br></p>
 <div><table border=0 cellpadding=1 cellspacing=2>
 <tr valign=top>
 <td width=12 style="background-color: #0000ff;">
 <p><span class=rvts6>&gt;</span></p>
 </td>
 <td width=737 style="background-color: #ffffff;">
 <p><span class=rvts7>kill -9 &nbsp;doesn't want to play either?</span></p>
 <p><br></p>
 <p><span class=rvts7>Read the docs/debugging file. Compile FR with debugging/developer stuff and run it under gdbm control and do your killing of firebird again.</span></p>
 <p><br></p>
 <p><span class=rvts7>It's likely to be the PERL integration as thats what will get done in when firebird is restarted. Are you perl with threading?</span></p>
 <p><br></p>
 <p><span class=rvts7>Not sure why your going through hoops for firebird access, I'm sure there's a native module....</span></p>
 <p><br></p>
 <p><span class=rvts7>alan</span></p>
 <p><br></p>
 <p><br></p>
 </td>
 </tr>
 </table>
 </div>
 <p><br></p>
 <p><br></p>
 <p># pkg_info | grep fire</p>
 <p>firebird-client-2.5.1 Firebird-2 database client</p>
 <p>firebird-server-2.5.1 Firebird-2 relational database (server)</p>
 <p># pkg_info | grep radi</p>
 <p>freeradius-2.1.12 &nbsp; A free RADIUS server implementation</p>
 <p><br></p>
 <p># perl -V</p>
 <p>Summary of my perl5 (revision 5 version 14 subversion 1) configuration:</p>
 <p><br></p>
 <p>&nbsp; Platform:</p>
 <p>&nbsp; &nbsp; osname=freebsd, osvers=9.0-current, archname=i386-freebsd-thread-multi-64int</p>
 <p>&nbsp; &nbsp; uname='freebsd flux 9.0-current freebsd 9.0-current #4: fri jun 10 01:30:12 utc 2011 adm@flux:usrobjusrsrcsyspae_kes i386 '</p>
 <p>&nbsp; &nbsp; config_args='-sde -Dprefix=/usr/local -Darchlib=/usr/local/lib/perl5/5.14.1/mach -Dprivlib=/usr/local/lib/perl5/5.14.1 -Dman3dir=/usr/local/lib/perl5/5.14.1/perl/man/man3 -Dman1dir=/usr/local/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/5.14.1/mach -Dsitelib=/usr/local/lib/perl5/site_perl/5.14.1 -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/5.14.1/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dcc=cc -Duseshrplib -Dinc_version_lis t=none -Dccflags=-DAPPLLIB_EXP="/usr/local/lib/perl5/5.14.1/BSDPAN" -Doptimize=-O2 -pipe -fno-strict-aliasing -Ui_gdbm -Dusethreads=y -Dusemymalloc=n -Duse64bitint'</p>
 <p>&nbsp; &nbsp; hint=recommended, useposix=true, d_sigaction=define</p>
 <p>&nbsp; &nbsp; useithreads=define, usemultiplicity=define</p>
 <p>&nbsp; &nbsp; useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef</p>
 <p>&nbsp; &nbsp; use64bitint=define, use64bitall=undef, uselongdouble=undef</p>
 <p>&nbsp; &nbsp; usemymalloc=n, bincompat5005=undef</p>
 <p>&nbsp; Compiler:</p>
 <p>&nbsp; &nbsp; cc='cc', ccflags ='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.14.1/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',</p>
 <p>&nbsp; &nbsp; optimize='-O2 -pipe -fno-strict-aliasing',</p>
 <p>&nbsp; &nbsp; cppflags='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.14.1/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'</p>
 <p>&nbsp; &nbsp; ccversion='', gccversion='4.2.2 20070831 prerelease [FreeBSD]', gccosandvers=''</p>
 <p>&nbsp; &nbsp; intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678</p>
 <p>&nbsp; &nbsp; d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12</p>
 <p>&nbsp; &nbsp; ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8</p>
 <p>&nbsp; &nbsp; alignbytes=4, prototype=define</p>
 <p>&nbsp; Linker and Libraries:</p>
 <p>&nbsp; &nbsp; ld='cc', ldflags ='-pthread -Wl,-E &nbsp;-fstack-protector -L/usr/local/lib'</p>
 <p>&nbsp; &nbsp; libpth=/usr/lib /usr/local/lib</p>
 <p>&nbsp; &nbsp; libs=-lgdbm -lm -lcrypt -lutil</p>
 <p>&nbsp; &nbsp; perllibs=-lm -lcrypt -lutil</p>
 <p>&nbsp; &nbsp; libc=, so=so, useshrplib=true, libperl=libperl.so</p>
 <p>&nbsp; &nbsp; gnulibc_version=''</p>
 <p>&nbsp; Dynamic Linking:</p>
 <p>&nbsp; &nbsp; dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' &nbsp;-Wl,-R/usr/local/lib/perl5/5.14.1/mach/CORE'</p>
 <p>&nbsp; &nbsp; cccdlflags='-DPIC -fPIC', lddlflags='-shared &nbsp;-L/usr/local/lib -fstack-protector'</p>
 <p><br></p>
 <p><br></p>
 <p>Characteristics of this binary (from libperl):</p>
 <p>&nbsp; Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV</p>
 <p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP</p>
 <p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; PERL_PRESERVE_IVUV USE_64_BIT_INT USE_ITHREADS</p>
 <p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF</p>
 <p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; USE_REENTRANT_API</p>
 <p>&nbsp; Built under freebsd</p>
 <p>&nbsp; Compiled at Dec &nbsp;9 2011 03:57:41</p>
 <p>&nbsp; @INC:</p>
 <p>&nbsp; &nbsp; /usr/local/lib/perl5/5.14.1/BSDPAN</p>
 <p>&nbsp; &nbsp; /usr/local/lib/perl5/site_perl/5.14.1/mach</p>
 <p>&nbsp; &nbsp; /usr/local/lib/perl5/site_perl/5.14.1</p>
 <p>&nbsp; &nbsp; /usr/local/lib/perl5/5.14.1/mach</p>
 <p>&nbsp; &nbsp; /usr/local/lib/perl5/5.14.1</p>
 <p>&nbsp; &nbsp; .</p>
 <p><br></p>
 <p>firebird is restarted without any problem.</p>
 <p>also if I ran after that 'radiusd -X' it is works fine too</p>
 <p># ps ax|grep rad</p>
 <p>&nbsp;9438 &nbsp;?? &nbsp;TLs &nbsp; &nbsp; 5:10.12 /usr/local/sbin/radiusd</p>
 <p>37170 &nbsp; 3 &nbsp;I+ &nbsp; &nbsp; &nbsp;0:00.33 radiusd -X</p>
 <p><br></p>
 <p>but, as you see, &nbsp;old 'radiusd' process is in state STOP and I can not do anything with them.</p>
 <p>As I have said erlier, even when I '#reboot' server hangsup waiting 'radiusd' for die</p>
 <p>Only hard reboot helps.</p>
 <p><br></p>
 <p>Maybe I do not understand correct, but it seems problem with a 'virtual memory' (vm_??):</p>
 <p>&nbsp; vm_map_lookup vm_fault_hold vm_fault trap_pfault trap calltrap</p>
 <p><br></p>
 <p>PS. in 'radiusd -X' mode bug is not repeatable.</p>
 <p><br></p>
 <p><span class=rvts8>--&nbsp;</span></p>
 <p><span class=rvts8>С уважением,</span></p>
 <p><span class=rvts8>&nbsp;Коньков &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;</span><a class=rvts9 href="mailto:kes-kes@yandex.ru">mailto:kes-kes@yandex.ru</a></p>
 
 </body></html>
 

From: Jilles Tjoelker <jilles@stack.nl>
To: =?utf-8?B?0JrQvtC90YzQutC+0LIg0JXQstCz0LXQvdC40Lk=?= <kes-kes@yandex.ru>
Cc: bug-followup@FreeBSD.org, freeradius-users@lists.freeradius.org,
	firebird-devel@lists.sourceforge.net
Subject: Re: bin/164526: kill(1) can not kill process despite on -KILL
Date: Thu, 2 Feb 2012 00:46:47 +0100

 On Thu, Feb 02, 2012 at 12:16:39AM +0200, Коньков Евгений wrote:
 > repeated again:
 > bug is repeateable:
 > 1. radiusd + mod_perl + example.pl(it is connects to FireBird) +
 > FireBIrd
 > 2. restart firebird
 > 3. try to restart radiusd
 > 4. process in fall into STOP state
 
 > # ps awx | grep radi
 >  9438  ??  TLs     5:10.12 /usr/local/sbin/radiusd
 > 27603   2  S+      0:00.00 grep radi
 > # procstat -k 9438
 >   PID    TID COMM             TDNAME           KSTACK
 >  9438 100080 radiusd          -                mi_switch sleepq_switch sleepq_wait _sx_xlock_hard _sx_xlock _vm_map_lock_upgrade vm_map_lookup vm_fault_hold vm_fault trap_pfault trap calltrap
 >  9438 100195 radiusd          -                mi_switch sleepq_switch sleepq_wait __lockmgr_args ffs_lock VOP_LOCK1_APV _vn_lock vm_object_deallocate unlock_and_deallocate vm_fault_hold vm_fault trap_pfault trap calltrap
 >  9438 101144 radiusd          -                mi_switch thread_suspend_switch thread_single exit1 sigexit postsig ast doreti_ast
 > # ps wHl9438
 >   UID   PID  PPID CPU PRI NI    VSZ    RSS MWCHAN STAT  TT     TIME COMMAND
 >   133  9438     1   0  20  0 351124 322000 user m TLs   ??  0:03.65 /usr/local/sbin/radiusd
 >   133  9438     1   0  20  0 351124 322000 ufs    TLs   ??  0:00.00 /usr/local/sbin/radiusd
 >   133  9438     1   0  20  0 351124 322000 -      TLs   ??  0:05.28 /usr/local/sbin/radiusd
 
 > if I can supply another usefull debug info, answer as fast as you can, I can
 > not wait too long. Thank you.
 
 OK, this looks like it may be useful for someone who knows more about
 the VM system than I do. It is very likely a FreeBSD kernel bug though,
 so building freeradius and/or firebird with debug information is
 unlikely to be useful (apart from perturbing a race condition, if the
 problem is related to a race condition).
 
 My analysis: thread 101144 is attempting to shut down the process in
 response to a signal, but needs to wait for 100080 and 100195 to finish
 page fault processing. For thread 100195, page fault processing resulted
 in deallocating a VM object based on some sort of file, and it is
 blocked waiting on the vnode lock for the file. It may or may not hold a
 lock on a user map. Thread 100080 needs to lock a user map to continue
 processing (this means the fault is either a copy-on-write fault or the
 first write to anonymous memory). It seems that 100080 is not holding
 the vnode lock that 100195 needs.
 
 If you have DDB (kernel debugger) and witness compiled in, the DDB
 command
   show locks
 will show who owns these locks. This is probably
 
 The output of
   procstat -kka
 may be useful (like the previous procstat command but for all threads in
 the system and with offsets from each function).
 
 The output of
   procstat -v 9438
 is the memory mappings of the process. It could be that this command
 gets stuck because of the locks.
 
 -- 
 Jilles Tjoelker

From: Alan DeKok <aland@deployingradius.com>
To: =?UTF-8?B?0JrQvtC90YzQutC+0LIg0JXQstCz0LXQvdC40Lk=?=
 <kes-kes@yandex.ru>, 
 FreeRadius users mailing list <freeradius-users@lists.freeradius.org>
Cc: Jilles Tjoelker <jilles@stack.nl>, 
 firebird-devel@lists.sourceforge.net, bug-followup@FreeBSD.org
Subject: Re: bin/164526: kill(1) can not kill process despite on -KILL
Date: Thu, 02 Feb 2012 08:13:54 +0100

 Коньков Евгений wrote:
 > repeated again:
 > bug is repeateable:
 > 1. radiusd + mod_perl + example.pl(it is connects to FireBird) +
 
   Why?  FreeRADIUS has native support for all major SQL servers.
 There's no need to use a Perl plugin.
 
 > FireBIrd
 > 2. restart firebird
 > 3. try to restart radiusd
 > 4. process in fall into STOP state
 
   You've built a system which has a lot of components.  The problem
 could be anywhere.
 
   I'll note that I've *never* seen this problem when using the native
 SQL plugins which are shipped with FreeRADIUS.
 
   Alan DeKok.

From: Andriy Gapon <avg@FreeBSD.org>
To: bug-followup@FreeBSD.org, kes-kes@yandex.ru
Cc:  
Subject: Re: bin/164526: kill(1) can not kill process despite on -KILL
Date: Thu, 02 Feb 2012 23:29:13 +0200

 Eugen,
 
 thank you for reporting and debugging this problem.
 In addition to what Jilles has already suggested I would like to also advise
 that it's possible to use kgdb to examine the vnode and its lock.
 You can use kgdb's 'tid' command to switch to a thread of interest (it would be
 100195 for your earlier report) and the you can use standard gdb commands to
 examine the data.
 
 Another, and more standard way, to deal with deadlocks like this one is
 described here:
 http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kerneldebug-deadlocks.html
 
 -- 
 Andriy Gapon
>Unformatted:
