From nobody@FreeBSD.org  Tue Mar 17 17:13:46 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CDECF106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 17 Mar 2009 17:13:46 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id BAED18FC08
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 17 Mar 2009 17:13:46 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n2HHDjaP089083
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 17 Mar 2009 17:13:45 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n2HHDj3U089081;
	Tue, 17 Mar 2009 17:13:45 GMT
	(envelope-from nobody)
Message-Id: <200903171713.n2HHDj3U089081@www.freebsd.org>
Date: Tue, 17 Mar 2009 17:13:45 GMT
From: Alexey Illarionov <littlesavage@orionet.ru>
To: freebsd-gnats-submit@FreeBSD.org
Subject: panic in net/if_mib.c
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         132734
>Category:       kern
>Synopsis:       [ifmib] [panic] panic in net/if_mib.c
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-net
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Mar 17 17:20:05 UTC 2009
>Closed-Date:    
>Last-Modified:  Mon May  4 16:30:01 UTC 2009
>Originator:     Alexey Illarionov
>Release:        7.1-STABLE Mar  8 2009
>Organization:
>Environment:
FreeBSD test.orionet.ru 7.1-STABLE FreeBSD 7.1-STABLE #11: Sun Mar  8 22:25:30 MSK 2009 littlesavage@test.orionet.ru:/usr/obj/usr/src/sys/VPN  i386
net.inet.ip.fastforwarding=1
net.isr.direct=1
>Description:
bsnmpd(8) with mibII module on a machine with many dynamic interfaces can lead to panic in net/if_mib.c

I have mpd5 with about 600 active pptp tunnel interfaces.

kernel dump:
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x68
fault code		= supervisor read, page not present
instruction pointer	= 0x20:0xc06dcaf6
stack pointer	        = 0x28:0xe64c3aac
frame pointer	        = 0x28:0xe64c3b54
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 27218 (bsnmpd)
trap number		= 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper(c09053a2,e64c3948,c063c0c9,c09217df,0,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c09217df,0,c08e5975,e64c3954,0,...) at kdb_backtrace+0x29
panic(c08e5975,c0922ab1,c5fa34dc,1,1,...) at panic+0x119
trap_fatal(c3fc484c,0,1,0,c09fc4c4,...) at trap_fatal+0x333
trap_pfault(c0a039ec,e64c39f4,c0895295,c09fbe80,c,...) at trap_pfault+0x250
trap(e64c3a6c) at trap+0x3c2
calltrap() at calltrap+0x6
--- trap 0xc, eip = 0xc06dcaf6, esp = 0xe64c3aac, ebp = 0xe64c3b54 ---
sysctl_ifdata(c09bbfe0,e64c3c24,2,e64c3ba4,e64c3ba4,...) at sysctl_ifdata+0x196
sysctl_root(e64c3ba4,e64c3bb0,4,c44cd8c0,c4c7b688,...) at sysctl_root+0x127
userland_sysctl(c44cd8c0,e64c3c14,6,0,bfbf8208,...) at userland_sysctl+0x134
__sysctl(c44cd8c0,e64c3cfc,18,6,c065b6ab,...) at __sysctl+0xde
syscall(e64c3d38) at syscall+0x335
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (202, FreeBSD ELF32, __sysctl), eip = 0x281871ef, esp = 0xbfbf7cfc, ebp = 0xbfbf7d28 ---
Uptime: 8d11h14m12s
Physical memory: 1012 MB
Dumping 282 MB: 267 251 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11

Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ichwd.ko
Reading symbols from /boot/modules/ng_ipacct.ko...done.
Loaded symbols for /boot/modules/ng_ipacct.ko
Reading symbols from /boot/kernel/ng_tee.ko...Reading symbols from /boot/kernel/ng_tee.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_tee.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
#0  doadump () at pcpu.h:196
196	pcpu.h: No such file or directory.
	in pcpu.h

(kgdb) bt
#0  doadump () at pcpu.h:196
#1  0xc063be1c in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc063c102 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3  0xc08a7673 in trap_fatal (frame=0xe64c3a6c, eva=104) at /usr/src/sys/i386/i386/trap.c:939
#4  0xc08a78d0 in trap_pfault (frame=0xe64c3a6c, usermode=0, eva=104) at /usr/src/sys/i386/i386/trap.c:852
#5  0xc08a82a2 in trap (frame=0xe64c3a6c) at /usr/src/sys/i386/i386/trap.c:530
#6  0xc088c90b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7  0xc06dcaf6 in sysctl_ifdata (oidp=0xc09bbfe0, arg1=0xe64c3c24, arg2=2, req=0xe64c3ba4)
    at /usr/src/sys/net/if_mib.c:127
#8  0xc0645897 in sysctl_root (oidp=Variable "oidp" is not available.
) at /usr/src/sys/kern/kern_sysctl.c:1307
#9  0xc06459e4 in userland_sysctl (td=0xc44cd8c0, name=0xe64c3c14, namelen=6, old=0x0, oldlenp=0xbfbf8208, 
    inkernel=0, new=0x0, newlen=0, retval=0xe64c3c10, flags=0) at /usr/src/sys/kern/kern_sysctl.c:1402
#10 0xc064677e in __sysctl (td=0xc44cd8c0, uap=0xe64c3cfc) at /usr/src/sys/kern/kern_sysctl.c:1337
#11 0xc08a7c25 in syscall (frame=0xe64c3d38) at /usr/src/sys/i386/i386/trap.c:1090
#12 0xc088c970 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
#13 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) up 7
#7  0xc06dcaf6 in sysctl_ifdata (oidp=0xc09bbfe0, arg1=0xe64c3c24, arg2=2, req=0xe64c3ba4)
    at /usr/src/sys/net/if_mib.c:127
127			ifp->if_snd.ifq_drops = ifmd.ifmd_snd_drops;

(kgdb) l
122			DONTCOPY(baudrate);
123	#undef DONTCOPY
124	#define COPY(fld) ifp->if_##fld = ifmd.ifmd_##fld
125			COPY(data);
126			ifp->if_snd.ifq_maxlen = ifmd.ifmd_snd_maxlen;
127			ifp->if_snd.ifq_drops = ifmd.ifmd_snd_drops;
128	#undef COPY
129			break;
130	
131		case IFDATA_LINKSPECIFIC:

(kgdb) p ifp
$1 = (struct ifnet *) 0x0
(kgdb) p *(int *)arg1
$4 = 332
(kgdb) p ifindex_table[332]
$12 = {ife_ifnet = 0x0, ife_dev = 0x0}


>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Mar 18 03:23:58 UTC 2009 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=132734 

From: Mikolaj Golub <to.my.trociny@gmail.com>
To: Alexey Illarionov <littlesavage@orionet.ru>
Cc: bug-followup@FreeBSD.org, Robert Watson <rwatson@FreeBSD.org>
Subject: Re: kern/132734: panic in net/if_mib.c
Date: Thu, 23 Apr 2009 22:29:36 +0300

 SVN rev 191435 on 2009-04-23 18:23:08Z by rwatson
 
 Merge r191434 from stable/7 to releng/7.2:
 
   In sysctl_ifdata(), query the ifnet pointer using the index only
   once, rather than querying it, validating it, and then re-querying
   it without validating it.  This may avoid a NULL pointer
   dereference and resulting kernel page fault if an interface is
   being deleted while bsnmp or other tools are querying data on the
   interface.
 
   The full fix, to properly refcount the interface for the duration
   of the sysctl, is in 8.x, but is considered too high-risk for
   7.2, so instead will appear in 7.3 (if all goes well).
 
 So, Alexey, can you try upgrading to the latest stable/7 or releng/7.2 or
 apply attached patch to see if this tweak at least eliminates the instant
 panic?
 
 --- if_mib.c	(revision 191424)
 +++ if_mib.c	(working copy)
 @@ -82,11 +82,9 @@
  		return EINVAL;
 
  	if (name[0] <= 0 || name[0] > if_index ||
 -	    ifnet_byindex(name[0]) == NULL)
 +	    (ifp = ifnet_byindex(name[0])) == NULL)
  		return ENOENT;
 
 -	ifp = ifnet_byindex(name[0]);
 -
  	switch(name[1]) {
  	default:
  		return ENOENT;

From: Robert Watson <rwatson@FreeBSD.org>
To: Mikolaj Golub <to.my.trociny@gmail.com>
Cc: Alexey Illarionov <littlesavage@orionet.ru>, bug-followup@FreeBSD.org
Subject: Re: kern/132734: panic in net/if_mib.c
Date: Thu, 23 Apr 2009 20:33:43 +0100 (BST)

 On Thu, 23 Apr 2009, Mikolaj Golub wrote:
 
 > SVN rev 191435 on 2009-04-23 18:23:08Z by rwatson
 >
 > Merge r191434 from stable/7 to releng/7.2:
 >
 >  In sysctl_ifdata(), query the ifnet pointer using the index only
 >  once, rather than querying it, validating it, and then re-querying
 >  it without validating it.  This may avoid a NULL pointer
 >  dereference and resulting kernel page fault if an interface is
 >  being deleted while bsnmp or other tools are querying data on the
 >  interface.
 >
 >  The full fix, to properly refcount the interface for the duration
 >  of the sysctl, is in 8.x, but is considered too high-risk for
 >  7.2, so instead will appear in 7.3 (if all goes well).
 >
 > So, Alexey, can you try upgrading to the latest stable/7 or releng/7.2 or 
 > apply attached patch to see if this tweak at least eliminates the instant 
 > panic?
 
 I'll try to get the refcount fix into 7-STABLE in about two weeks, assuming no 
 hitches in the 8.x version.  This will close a number of related race 
 conditions, which we've had occasional reports of (and others that we 
 haven't).
 
 Robert N M Watson
 Computer Laboratory
 University of Cambridge

From: Alexey Illarionov <littlesavage@orionet.ru>
To: Mikolaj Golub <to.my.trociny@gmail.com>
Cc: bug-followup@FreeBSD.org, Robert Watson <rwatson@FreeBSD.org>
Subject: Re: kern/132734: panic in net/if_mib.c
Date: Mon, 04 May 2009 20:00:17 +0400

 Hi
 
 Mikolaj Golub wrote:
 > So, Alexey, can you try upgrading to the latest stable/7 or releng/7.2 or
 > apply attached patch to see if this tweak at least eliminates the instant
 > panic?
 
 With this patch this panic does not repeat any more.
 There are some error messages in log files:
 snmpd: sysctl linkmib estimate (ng1): No such file or directory.
 But kernel does not panics. Thanks.
>Unformatted:
