From nobody@FreeBSD.org  Tue Jan  2 13:57:40 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 0762316A40F
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  2 Jan 2007 13:57:40 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [69.147.83.33])
	by mx1.freebsd.org (Postfix) with ESMTP id EAB5C13C457
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  2 Jan 2007 13:57:39 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id l02DvcHQ003807
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 2 Jan 2007 13:57:38 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id l02Dvcl4003806;
	Tue, 2 Jan 2007 13:57:38 GMT
	(envelope-from nobody)
Message-Id: <200701021357.l02Dvcl4003806@www.freebsd.org>
Date: Tue, 2 Jan 2007 13:57:38 GMT
From: Michael Nottebrock<lofi@FreeBSD.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Regular kernel panics related to ipv6 interface management/manipulation
X-Send-Pr-Version: www-3.0

>Number:         107431
>Category:       kern
>Synopsis:       [ipv6] Regular kernel panics related to ipv6 interface management/manipulation
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 02 14:00:35 GMT 2007
>Closed-Date:    Sat Jan 26 18:33:27 UTC 2008
>Last-Modified:  Sat Jan 26 18:33:27 UTC 2008
>Originator:     Michael Nottebrock
>Release:        6.1-RELEASE-p10
>Organization:
>Environment:
FreeBSD lofi.dyndns.org 6.1-RELEASE-p10 FreeBSD 6.1-RELEASE-p10 #8: Sun Nov 19 05:15:03 CET 2006     lofi@lofi.dyndns.org:/usr/obj/usr/src/sys/LOFI.ULE  i386
>Description:
I have been getting this and similar kernel panics on a regular, but overall rare basis for a very long time, in FreeBSD 6.0-RELEASE and 6.1-RELEASE. However, I've only recently been able to turn on crashdumps and thus submit this bug report.

The machine has an ipv6 tunnel connection via freenet6, managed by freenet6's tunnel setup utility, tspc (in FreeBSD ports under net/freenet6). It is connected to the internet over a DSL line, with an automatic disconnect every 24 hours. The ipv6 tunnel interface is destroyed before the ipv4 connection goes down and re-setup once the ipv4 connection is back up. The panics however do not always occur around that time, this last one which I pasted below for instance occurred about five hours after the DSL "redial". Traffic on the ipv6 tunnel is very moderate, but constant (ntp).

Panic message and backtrace, gathered from crashdump with kgdb:

[lofi@lofi]:0:/home/lofi # kgdb /usr/obj/usr/src/sys/LOFI.ULE/kernel.debug /usr/crash/vmcore.79
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x6c6983fc
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0xc0567a63
stack pointer           = 0x28:0xdac3ab70
frame pointer           = 0x28:0xdac3ab80
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 2842 (ifconfig)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 11d23h49m19s
Dumping 511 MB (2 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 511MB (130800 pages) 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15

#0  doadump () at pcpu.h:165
165     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:165
#1  0xc04f8558 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:402
#2  0xc04f8881 in panic (fmt=0xc06b993e "%s") at /usr/src/sys/kern/kern_shutdown.c:558
#3  0xc068f140 in trap_fatal (frame=0xdac3ab30, eva=1818854396) at /usr/src/sys/i386/i386/trap.c:836
#4  0xc068ee7f in trap_pfault (frame=0xdac3ab30, usermode=0, eva=1818854396) at /usr/src/sys/i386/i386/trap.c:744
#5  0xc068ead9 in trap (frame=
      {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = 1818853760, tf_esi = 1920169263, tf_ebp = -624710784, tf_isp = -624710820, tf_ebx = -999521920, tf_edx = -1008841600, tf_ecx = 4, tf_eax = 4, tf_trapno = 12, tf_err = 2, tf_eip = -1068074397, tf_cs = 32, tf_eflags = 66178, tf_esp = -1067635198, tf_ss = -999521920}) at /usr/src/sys/i386/i386/trap.c:434
#6  0xc067c3da in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc0567a63 in if_delmulti (ifp=0x6c698180, sa=0x7273752f) at atomic.h:146
#8  0xc05cd1fb in in6_delmulti (in6m=0xc472a280) at /usr/src/sys/netinet6/mld6.c:649
#9  0xc05c09f8 in in6_ifdetach (ifp=0xc37bd400) at /usr/src/sys/netinet6/in6_ifattach.c:806
#10 0xc0565485 in if_detach (ifp=0xc37bd400) at /usr/src/sys/net/if.c:658
#11 0xc056a7e4 in gif_destroy (sc=0xc5737780) at /usr/src/sys/net/if_gif.c:209
#12 0xc056a896 in gif_clone_destroy (ifp=0x4) at /usr/src/sys/net/if_gif.c:226
#13 0xc0568d9e in ifc_simple_destroy (ifc=0xc070adc0, ifp=0x4) at /usr/src/sys/net/if_clone.c:478
#14 0xc05683c9 in if_clone_destroy (name=0xc3e21940 "gif0") at /usr/src/sys/net/if_clone.c:172
#15 0xc0566f5c in ifioctl (so=0xc4fae6f4, cmd=2149607801, data=0xc3e21940 "gif0", td=0xc3de4c80) at /usr/src/sys/net/if.c:1508
#16 0xc05225c3 in soo_ioctl (fp=0x4, cmd=2149607801, data=0xc3e21940, active_cred=0xc3989900, td=0xc3de4c80) at /usr/src/sys/kern/sys_socket.c:214
#17 0xc051ca2d in ioctl (td=0xc3de4c80, uap=0xdac3ad04) at file.h:258
#18 0xc068f487 in syscall (frame=
      {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = -1077941088, tf_esi = -1077940864, tf_ebp = -1077943304, tf_isp = -624710300, tf_ebx = 134570752, tf_edx = 134581885, tf_ecx = 0, tf_eax = 54, tf_trapno = 12, tf_err = 2, tf_eip = 672368511, tf_cs = 51, tf_eflags = 642, tf_esp = -1077943332, tf_ss = 59})
    at /usr/src/sys/i386/i386/trap.c:981
#19 0xc067c42f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200
#20 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)

>How-To-Repeat:
Not easily - I haven't found a way to trigger the problem at will. Given that freenet6's tunnel is not very stable, and thus assuming that the gif0 interface probably goes up and down twice a day on average (counting once for the cronjob on DSL-redial and counting once for random service outage), my guess is that roundabout every 30-40 'ifconfig gif0 up/down' (not sure what tspc really does with it) invocations, the machine will panic.
>Fix:

>Release-Note:
>Audit-Trail:

From: Alexander Motin <mav@alkar.net>
To: bug-followup@FreeBSD.org,  lofi@FreeBSD.org
Cc:  
Subject: Re: kern/107431: [ipv6] Regular kernel panics related to ipv6 interface
 management/manipulation
Date: Mon, 19 Mar 2007 01:44:34 +0200

 I am regularly observe problem with smething alike simptoms. I have 
 FreeBSD 6.2-STABLE of Jan 29. I have IPv6 in my kernel, but do not use 
 it actively. In my case it happends with significant probability when 
 mpd4.1 based server trying to destroy several ngX interfaces on 
 shutdown. It does it by shutting down related ng_iface netgraph node.
 
 Fatal trap 12: page fault while in kernel mode
 fault virtual address   = 0x100027c
 fault code              = supervisor write, page not present
 instruction pointer     = 0x20:0xc05df5a3
 stack pointer           = 0x28:0xdce8c94c
 frame pointer           = 0x28:0xdce8c970
 code segment            = base 0x0, limit 0xfffff, type 0x1b
                          = DPL 0, pres 1, def32 1, gran 1
 processor eflags        = interrupt enabled, resume, IOPL = 0
 current process         = 6089 (mpd4)
 trap number             = 12
 panic: page fault
 Uptime: 4h43m35s
 Dumping 511 MB (2 chunks)
    chunk 0: 1MB (159 pages) ... ok
    chunk 1: 511MB (130800 pages) 495 479 463 447 431 415 399 383 367 351 
 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 
 31 15
 
 #0  doadump () at pcpu.h:165
 165             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
 (kgdb) bt
 #0  doadump () at pcpu.h:165
 #1  0xc055e046 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
 #2  0xc055e350 in panic (fmt=0xc0749735 "%s") at 
 /usr/src/sys/kern/kern_shutdown.c:565
 #3  0xc0723095 in trap_fatal (frame=0xdce8c90c, eva=0) at 
 /usr/src/sys/i386/i386/trap.c:837
 #4  0xc0722db5 in trap_pfault (frame=0xdce8c90c, usermode=0, 
 eva=16777852) at /usr/src/sys/i386/i386/trap.c:745
 #5  0xc072299f in trap (frame=
        {tf_fs = -588775416, tf_es = -1068171224, tf_ds = -588775384, 
 tf_edi = 16777216, tf_esi = 167772927, tf_ebp = -588723856, tf_isp = 
 -588723912, tf_ebx = -1008249152, tf_edx = -1011626624, tf_ecx = 
 -1007975136, tf_eax = 4, tf_trapno = 12, tf_err = 2, tf_eip = 
 -1067584093, tf_cs = 32, tf_eflags = 66194, tf_esp = -1015311360, tf_ss 
 = -2145359566}) at /usr/src/sys/i386/i386/trap.c:435
 #6  0xc070fb5a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
 #7  0xc05df5a3 in if_delmulti (ifp=0x1000000, sa=0xa0002ff) at atomic.h:146
 #8  0xc05f03cd in in_delmulti_locked (inm=0xc3eb8520) at 
 /usr/src/sys/netinet/in.c:1060
 #9  0xc05f049b in in_delmulti_ifp (ifp=0xc37b9400) at 
 /usr/src/sys/netinet/in.c:1079
 #10 0xc05f0568 in in_ifdetach (ifp=0xc37b9400) at 
 /usr/src/sys/netinet/in.c:1095
 #11 0xc05dc82b in if_detach (ifp=0xc37b9400) at /usr/src/sys/net/if.c:655
 
 This looks strange for me:
 (kgdb) frame 8
 #8  0xc05f03cd in in_delmulti_locked (inm=0xc3eb8520) at 
 /usr/src/sys/netinet/in.c:1060
 1060            if_delmulti(ifma->ifma_ifp, ifma->ifma_addr);
 (kgdb) p ifma->ifma_ifp
 $8 = (struct ifnet *) 0x1000000
 (kgdb) p *(ifma->ifma_ifp)
 Cannot access memory at address 0x1000000
 
 I also have several other alike coredumps:
 
 #6  0xc070fb5a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
 #7  0xc05df5a3 in if_delmulti (ifp=0x80000, sa=0x0) at atomic.h:146
 #8  0xc05f03cd in in_delmulti_locked (inm=0xc4a3e7c0) at 
 /usr/src/sys/netinet/in.c:1060
 #9  0xc05f049b in in_delmulti_ifp (ifp=0xc385fc00) at 
 /usr/src/sys/netinet/in.c:1079
 #10 0xc05f0568 in in_ifdetach (ifp=0xc385fc00) at 
 /usr/src/sys/netinet/in.c:1095
 #11 0xc05dc82b in if_detach (ifp=0xc385fc00) at /usr/src/sys/net/if.c:655
 
 ----
 #5  0xc070fb5a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
 #6  0xc05839e5 in turnstile_setowner (ts=0xc3a2fcc0, owner=0x4) at 
 /usr/src/sys/kern/subr_turnstile.c:434
 #7  0xc0583d11 in turnstile_wait (lock=0xc385e660, owner=0x4) at 
 /usr/src/sys/kern/subr_turnstile.c:593
 #8  0xc0553aeb in _mtx_lock_sleep (m=0xc385e660, tid=3286708992, opts=0, 
 file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:579
 #9  0xc05df5df in if_delmulti (ifp=0xc385e400, sa=0xc3e79b80) at 
 /usr/src/sys/net/if.c:2083
 #10 0xc05f03cd in in_delmulti_locked (inm=0x4) at 
 /usr/src/sys/netinet/in.c:1060
 #11 0xc05f049b in in_delmulti_ifp (ifp=0xc3855000) at 
 /usr/src/sys/netinet/in.c:1079
 #12 0xc05f0568 in in_ifdetach (ifp=0xc3855000) at 
 /usr/src/sys/netinet/in.c:1095
 #13 0xc05dc82b in if_detach (ifp=0xc3855000) at /usr/src/sys/net/if.c:655
 
 ---
 #6  0xc070fb5a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
 #7  0xc05df5a3 in if_delmulti (ifp=0x0, sa=0x50001ff) at atomic.h:146
 #8  0xc05f03cd in in_delmulti_locked (inm=0xc50901c0) at 
 /usr/src/sys/netinet/in.c:1060
 #9  0xc05f049b in in_delmulti_ifp (ifp=0xc4b1a800) at 
 /usr/src/sys/netinet/in.c:1079
 #10 0xc05f0568 in in_ifdetach (ifp=0xc4b1a800) at 
 /usr/src/sys/netinet/in.c:1095
 #11 0xc05dc82b in if_detach (ifp=0xc4b1a800) at /usr/src/sys/net/if.c:655
 
 If anybody needs additional info, I will be glad to help.
 
 -- 
 Alexander Motin
State-Changed-From-To: open->closed 
State-Changed-By: gavin 
State-Changed-When: Sat Jan 26 18:31:25 UTC 2008 
State-Changed-Why:  
Close, duplicate of kern/108197.  This is believed to be fixed in at least 
7.x, but so far hasn't been confirmed.  To submitter:  are you able to test 
that that is indeed the case and submit your followup to kern/108197? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=107431 
>Unformatted:
