From nobody@FreeBSD.org  Thu Nov  8 06:01:24 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 531F7AA3
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  8 Nov 2012 06:01:24 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 3A99A8FC18
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  8 Nov 2012 06:01:24 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.5/8.14.5) with ESMTP id qA861Ou2008129
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 8 Nov 2012 06:01:24 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.5/8.14.5/Submit) id qA861NdA008128;
	Thu, 8 Nov 2012 06:01:23 GMT
	(envelope-from nobody)
Message-Id: <201211080601.qA861NdA008128@red.freebsd.org>
Date: Thu, 8 Nov 2012 06:01:23 GMT
From: Rumen Palov <skarida@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD 9.1 restarts in random fashion after upgrade to RC2 or RC3, triggered from sysctl calls
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         173465
>Category:       amd64
>Synopsis:       FreeBSD 9.1 restarts in random fashion after upgrade to RC2 or RC3, triggered from sysctl calls
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-amd64
>State:          open
>Quarter:
>Keywords:
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Nov 08 06:10:00 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator:     Rumen Palov
>Release:        FreeBSD 9.2 RC3
>Organization:
E-CARD LTD.
>Environment:
FreeBSD web1 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #8: Mon Nov  5 17:20:11 MSK 2012     root@web1:/usr/obj/usr/src/sys/VBOX  amd64
>Description:
We one group of servers which are monitored via perl script. 
We take memory condition with sysctl call on each 10-20seconds. So if this script is working, the servers are restarted each nigth in random fashion. We stop this stats collecting and servers are working normal. We call sysctl -a.  
From 12 servers we have more than the same half restarted each night. 

I have perform test with calling sysctl-a for 1million times in loop, but this don't trigger kernel trap. 

Sysctl sometimes return message:  "sysctl: malloc failed".

core.txt:
---------------------------------------------------------------------------

Fatal trap 9: general protection fault while in kernel mode
cpuid = 5; apic id = 13
instruction pointer     = 0x20:0xffffffff805a8578
stack pointer           = 0x28:0xffffff8910fb5810
frame pointer           = 0x28:0xffffff8910fb5840
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 52704 (sysctl)
trap number             = 9
panic: general protection fault
cpuid = 5
KDB: stack backtrace:
#0 0xffffffff805f3df6 at kdb_backtrace+0x66
#1 0xffffffff805bde8e at panic+0x1ce
#2 0xffffffff808208b0 at trap_fatal+0x290
#3 0xffffffff80820f34 at trap+0x104
#4 0xffffffff8080ab0f at calltrap+0x8
#5 0xffffffff8066a2e9 at bpf_stats_sysctl+0x409
#6 0xffffffff805c78d7 at sysctl_root+0x157
#7 0xffffffff805c7bb5 at userland_sysctl+0x145
#8 0xffffffff805c80ca at sys___sysctl+0xaa
#9 0xffffffff80820190 at amd64_syscall+0x540
#10 0xffffffff8080adf7 at Xfast_syscall+0xf7


Reading symbols from /boot/kernel/if_carp.ko...Reading symbols from /boot/kernel/if_carp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/if_carp.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
#0  doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:229
229     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) #0  doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:229
#1  0xffffffff805bd964 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:448
#2  0xffffffff805bde67 in panic (fmt=0x1 <Address 0x1 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:636
#3  0xffffffff808208b0 in trap_fatal (frame=0x9, eva=Variable "eva" is not available.
)
    at /usr/src/sys/amd64/amd64/trap.c:878
#4  0xffffffff80820f34 in trap (frame=0xffffff8910fb5760)
    at /usr/src/sys/amd64/amd64/trap.c:605
#5  0xffffffff8080ab0f in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:228
#6  0xffffffff805a8578 in free (addr=0xfffffe05d5ab5c00,
    mtp=0xffffffff80b8e1e0) at /usr/src/sys/kern/kern_malloc.c:569
#7  0xffffffff8066a2e9 in bpf_stats_sysctl (oidp=Variable "oidp" is not available.
)
    at /usr/src/sys/net/bpf.c:2504
#8  0xffffffff805c78d7 in sysctl_root (oidp=Variable "oidp" is not available.
)
    at /usr/src/sys/kern/kern_sysctl.c:1513
#9  0xffffffff805c7bb5 in userland_sysctl (td=Variable "td" is not available.
)
    at /usr/src/sys/kern/kern_sysctl.c:1623
#10 0xffffffff805c80ca in sys___sysctl (td=0xfffffe0010f5c000,
    uap=0xffffff8910fb5bb0) at /usr/src/sys/kern/kern_sysctl.c:1549
#11 0xffffffff80820190 in amd64_syscall (td=0xfffffe0010f5c000, traced=0)
    at subr_syscall.c:135
#12 0xffffffff8080adf7 in Xfast_syscall ()
    at /usr/src/sys/amd64/amd64/exception.S:387
#13 0x0000000800924d7c in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)

>How-To-Repeat:
Call sysctl -a on each 10 seconds for 24 hours. 
>Fix:
We don't have fix. We just stop execution of this script to prevent restarting.

>Release-Note:
>Audit-Trail:
>Unformatted:
