From nobody@FreeBSD.org  Thu Feb  7 11:40:36 2008
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id AD31016A41A
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  7 Feb 2008 11:40:36 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 8EC4513C457
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  7 Feb 2008 11:40:36 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m17BcWvY099641
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 7 Feb 2008 11:38:32 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id m17BcVXK099640;
	Thu, 7 Feb 2008 11:38:31 GMT
	(envelope-from nobody)
Message-Id: <200802071138.m17BcVXK099640@www.freebsd.org>
Date: Thu, 7 Feb 2008 11:38:31 GMT
From: Paul Procacci <pprocacci@bellsouth.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [panic] Reproducible Crash - network interface related [kgdb output included]
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         120343
>Category:       kern
>Synopsis:       [panic] Reproducible Crash - network interface related (kgdb output included)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    ups
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Feb 07 11:50:00 UTC 2008
>Closed-Date:    Fri Mar 15 23:34:46 UTC 2013
>Last-Modified:  Fri Mar 15 23:34:46 UTC 2013
>Originator:     Paul Procacci
>Release:        6.3-RELEASE
>Organization:
DataPipe
>Environment:
FreeBSD nat.myhome.net 6.3-RELEASE FreeBSD 6.3-RELEASE #0: Thu Feb  7 04:45:27 CST 2008     root@nat.myhome.net:/usr/obj/usr/src/sys/FIREWALL.DEBUG  i386
>Description:
When I start and/or stop ushare for sharing my media amonst all machines
on my network, seconds later the kernel panics.  1/2 the time it's when
I start the service, and the other 1/2 when I stop it.  But in either
case, it will undoubtedly cause a panic.  I can provide more information
than below if needed.  Please just ask.

#################################################################
nat# kgdb kernel.debug /var/crash/vmcore.1
kgdb: kvm_nlist(_stopped_cpus):
kgdb: kvm_nlist(_stoppcbs):
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x65707573
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc056bc69
stack pointer           = 0x28:0xcd5e0920
frame pointer           = 0x28:0xcd5e0934
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 1858 (ushare)
trap number             = 12
panic: page fault
Uptime: 5m11s
Dumping 254 MB (2 chunks)
  chunk 0: 1MB (160 pages) ... ok
  chunk 1: 254MB (64942 pages) 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14

#0  doadump () at pcpu.h:165
165             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) list *0xc056bc69
0xc056bc69 is in if_findmulti (/usr/src/sys/net/if.c:1890).
1885
1886            IF_ADDR_LOCK_ASSERT(ifp);
1887
1888            TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
1889                    if (sa->sa_family == AF_LINK) {
1890                            if (sa_dl_equal(ifma->ifma_addr, sa))
1891                                    break;
1892                    } else {
1893                            if (sa_equal(ifma->ifma_addr, sa))
1894                                    break;
(kgdb) backtrace
#0  doadump () at pcpu.h:165
#1  0xc04e0a1c in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc04e0d69 in panic (fmt=0xc06ac332 "%s") at /usr/src/sys/kern/kern_shutdown.c:565
#3  0xc0688a9c in trap_fatal (frame=0xcd5e08e0, eva=0) at /usr/src/sys/i386/i386/trap.c:838
#4  0xc0688772 in trap_pfault (frame=0xcd5e08e0, usermode=0, eva=1701868915) at /usr/src/sys/i386/i386/trap.c:745
#5  0xc068832f in trap (frame=
      {tf_fs = -1068761080, tf_es = -1038680024, tf_ds = 40, tf_edi = 0, tf_esi = -1034427392, tf_ebp = -849475276, tf_isp = -849475316, tf_ebx = -1035861376, tf_edx = 1701868915, tf_ecx = -1033389824, tf_eax = 16, tf_trapno = 12, tf_err = 0, tf_eip = -1068057495, tf_cs = 32, tf_eflags = 66118, tf_esp = -849475188, tf_ss = -1056698368}) at /usr/src/sys/i386/i386/trap.c:435
#6  0xc0672a2a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc056bc69 in if_findmulti (ifp=0x10, sa=0xc257e400) at /usr/src/sys/net/if.c:1890
#8  0xc056c2b8 in if_delmulti_locked (ifma=0xc22c3140) at /usr/src/sys/net/if.c:2119
#9  0xc056c5b8 in if_delmulti_ent (ifma=0xc22c3140) at /usr/src/sys/net/if.c:2174
#10 0xc059a418 in in_delmulti_locked (inm=0xc25bac80, all=0) at /usr/src/sys/netinet/in.c:1049
#11 0xc059a500 in in_delmulti (inm=0xc25bac80) at /usr/src/sys/netinet/in.c:1066
#12 0xc05b030c in ip_freemoptions (imo=0xc249c800) at /usr/src/sys/netinet/ip_output.c:2064
#13 0xc059babb in in_pcbdetach (inp=0xc23d32d0) at /usr/src/sys/netinet/in_pcb.c:718
#14 0xc05c2a98 in udp_detach (so=0x10) at /usr/src/sys/netinet/udp_usrreq.c:1071
#15 0xc0527872 in soclose (so=0xc23a36f4) at /usr/src/sys/kern/uipc_socket.c:459
#16 0xc051342d in soo_close (fp=0xc2617360, td=0xc267b900) at /usr/src/sys/kern/sys_socket.c:317
#17 0xc04b9740 in fdrop_locked (fp=0xc2617360, td=0x10) at file.h:296
#18 0xc04b960f in fdrop (fp=0xc2617360, td=0x10) at /usr/src/sys/kern/kern_descrip.c:2113
#19 0xc04b7452 in closef (fp=0xc2617360, td=0xc267b900) at /usr/src/sys/kern/kern_descrip.c:1933
#20 0xc04b5f76 in fdfree (td=0xc267b900) at /usr/src/sys/kern/kern_descrip.c:1651
#21 0xc04c221a in exit1 (td=0xc267b900, rv=15) at /usr/src/sys/kern/kern_exit.c:273
#22 0xc04e618c in sigexit (td=0xc267b900, sig=15) at /usr/src/sys/kern/kern_sig.c:2459
#23 0xc04c9429 in kse_thr_interrupt (td=0xc267b900, uap=0xcd5e0d04) at /usr/src/sys/kern/kern_kse.c:239
#24 0xc0688e60 in syscall (frame=
      {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = 134693376, tf_esi = 3, tf_ebp = -1080037592, tf_isp = -849474204, tf_ebx = 672007348, tf_edx = 15, tf_ecx = 15, tf_eax = 382, tf_trapno = 12, tf_err = 2, tf_eip = 671995035, tf_cs = 51, tf_eflags = 514, tf_esp = -1080037732, tf_ss = 59})
    at /usr/src/sys/i386/i386/trap.c:984
#25 0xc0672a7f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200
#26 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
>How-To-Repeat:
Start and/or stop the service ushare.

Since this is if_* related, I feel compelled to provide my network
interfaces.  Whether or not it's relevant, I'm not sure, but here it is,
just in case.  If you need anything other than this, please ask.

################################################################
nat# ifconfig
ath0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 2290
        ether 00:17:9a:ba:73:5a
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
        status: associated
        ssid procacci_home channel 2 bssid 00:17:9a:ba:73:5a
        authmode WPA privacy MIXED deftxkey 2 TKIP 2:128-bit txpowmax 35
        bmiss 7 protmode CTS burst dtimperiod 1 bintval 100
fxp0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        ether 00:02:b3:cd:bb:f7
        media: Ethernet 100baseTX <full-duplex>
        status: active
de0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        ether 00:00:d1:1e:84:88
        media: Ethernet 100baseTX <full-duplex>
        status: no carrier
de1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet 10.5.21.241 netmask 0xfffffff8 broadcast 10.5.21.247
        ether 00:00:d1:1e:84:89
        media: Ethernet 100baseTX <full-duplex>
        status: active
de2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet 10.5.21.249 netmask 0xfffffff8 broadcast 10.5.21.255
        ether 00:00:d1:1e:84:8a
        media: Ethernet 100baseTX <full-duplex>
        status: active
de3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        ether 00:00:d1:1e:84:8b
        media: Ethernet 100baseTX <full-duplex>
        status: no carrier
xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=9<RXCSUM,VLAN_MTU>
        inet 192.168.1.97 netmask 0xffffff00 broadcast 255.255.255.255
        ether 00:b0:d0:87:d3:90
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
        inet 127.0.0.1 netmask 0xff000000
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet 10.5.21.1 netmask 0xffffff80 broadcast 10.5.21.127
        ether b6:56:f6:ee:74:c8
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto stp maxaddr 100 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: de3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
        member: de0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
        member: fxp0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
        member: ath0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1492
        inet 70.146.137.153 --> 68.216.204.65 netmask 0xffffff00
        Opened by PID 652
tun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1450
        inet 10.5.22.1 --> 10.5.22.2 netmask 0xff000000
        Opened by PID 1594
>Fix:
Unknown.

>Release-Note:
>Audit-Trail:

From: John Baldwin <jhb@FreeBSD.org>
To: bug-followup@FreeBSD.org, pprocacci@bellsouth.net
Cc: ups@FreeBSD.org
Subject: Re: kern/120343: [panic] Reproducible Crash - network interface related (kgdb output included)
Date: Mon, 25 Feb 2008 14:14:30 -0500

 Try this fix:
 
 ups         2008-02-22 19:13:57 UTC
 
   FreeBSD src repository
 
   Modified files:        (Branch: RELENG_6)
     sys/netinet          in.c 
   Log:
   Fix reference counting for already existing addresses in in_addmulti()
   
   Reviewed by:    gnn@
   
   Revision   Changes    Path
   1.85.2.10  +0 -1      src/sys/netinet/in.c
 
 -- 
 John Baldwin

From: "Paul A. Procacci" <pprocacci@datapipe.com>
To: bug-followup@FreeBSD.org, pprocacci@bellsouth.net
Cc:  
Subject: Re: kern/120343: [panic] Reproducible Crash - network interface related
 (kgdb output included)
Date: Mon, 25 Feb 2008 22:19:54 -0600

 Supped today @ 2/25/08 :: 8:00pm CST.  My kernel build error'd out with 
 the below.  That config file worked previously, so before I can report 
 back on whether what you provided worked or not, I need to get a new 
 kernel compiled first.  I'll let ya know when I do.
 
 Thanks,
 John!
 
 ###################################################################################################
 ###################################################################################################
 
 ===> unionfs (all)
 cc -O2 -pipe -march=pentium3 -fno-strict-aliasing -Werror -D_KERNEL 
 -DKLD_MODULE                                                                              
 -nostdinc -I-   -DHAVE_KERNEL_OPTION_HEADERS -include 
 /usr/obj/usr/src/sys/FIRE                                                                             
 WALL/opt_global.h -I. -I@ -I@/contrib/altq -I@/../include 
 -finline-limit=8000 
 -f                                                                             
 no-common  -I/usr/obj/usr/src/sys/FIREWALL -mno-align-long-strings 
 -mpreferred-s                                                                             
 tack-boundary=2  -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -ffreestanding 
 -Wall 
 -Wr                                                                             
 edundant-decls -Wnested-externs -Wstrict-prototypes  
 -Wmissing-prototypes 
 -Wpoin                                                                             
 ter-arith -Winline -Wcast-qual  -fformat-extensions -std=c99 -c 
 /usr/src/sys/mod                                                                             
 ules/unionfs/../../fs/unionfs/union_subr.c
 cc -O2 -pipe -march=pentium3 -fno-strict-aliasing -Werror -D_KERNEL 
 -DKLD_MODULE                                                                              
 -nostdinc -I-   -DHAVE_KERNEL_OPTION_HEADERS -include 
 /usr/obj/usr/src/sys/FIRE                                                                             
 WALL/opt_global.h -I. -I@ -I@/contrib/altq -I@/../include 
 -finline-limit=8000 
 -f                                                                             
 no-common  -I/usr/obj/usr/src/sys/FIREWALL -mno-align-long-strings 
 -mpreferred-s                                                                             
 tack-boundary=2  -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -ffreestanding 
 -Wall 
 -Wr                                                                             
 edundant-decls -Wnested-externs -Wstrict-prototypes  
 -Wmissing-prototypes 
 -Wpoin                                                                             
 ter-arith -Winline -Wcast-qual  -fformat-extensions -std=c99 -c 
 /usr/src/sys/mod                                                                             
 ules/unionfs/../../fs/unionfs/union_vfsops.c
 /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vfsops.c:443: error: 
 conflic                                                                             
 ting types for 'unionfs_quotactl'
 /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vfsops.c:58: error: 
 previous                                                                              
 declaration of 'unionfs_quotactl' was here
 /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vfsops.c:443: error: 
 conflic                                                                             
 ting types for 'unionfs_quotactl'
 /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vfsops.c:58: error: 
 previous                                                                              
 declaration of 'unionfs_quotactl' was here
 /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vfsops.c:560: 
 warning: 
 initi                                                                             
 alization from incompatible pointer type
 /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vfsops.c:58: 
 warning: 
 'union                                                                             
 fs_quotactl' used but never defined
 *** Error code 1
 
 Stop in /usr/src/sys/modules/unionfs.
 *** Error code 1
 
 Stop in /usr/src/sys/modules.
 *** Error code 1
 
 Stop in /usr/obj/usr/src/sys/FIREWALL.
 *** Error code 1
 
 Stop in /usr/src.
 *** Error code 1
 
 Stop in /usr/src.
 

From: Seth Kingsley <sethk@meowfishies.com>
To: bug-followup@FreeBSD.org, pprocacci@bellsouth.net
Cc:  
Subject: Re: kern/120343: [panic] Reproducible Crash - network interface
	related (kgdb output included)
Date: Sun, 28 Sep 2008 21:47:02 -0700

 I am running 6.3-R and was seeing intermittent crashes with the same backtrace
 while shutting down the bittorrent client.  Now after applying the patch and
 using the client over the course of several days I haven't crashed again.
State-Changed-From-To: open->patched 
State-Changed-By: linimon 
State-Changed-When: Mon Sep 29 09:26:02 UTC 2008 
State-Changed-Why:  
ups@ committed a patch. 

To submitter: does updating to this patch fix your problem?  There is 
an indication in a followup from a different user that it does. 


Responsible-Changed-From-To: freebsd-bugs->ups 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Sep 29 09:26:02 UTC 2008 
Responsible-Changed-Why:  

http://www.freebsd.org/cgi/query-pr.cgi?pr=120343 

From: "Paul A. Procacci" <pprocacci@datapipe.com>
To: bug-followup@FreeBSD.org, pprocacci@bellsouth.net
Cc:  
Subject: Re: kern/120343: [panic] Reproducible Crash - network interface related
 (kgdb output included)
Date: Mon, 29 Sep 2008 04:36:35 -0500

 Hello,
 
 I cannot confirm that the patch works.  I have since updated to FBSD 
 7.0-RELEASE on the machine in question.  Since supp'ing to 7.0 the 
 machine no longer behaved as described in my inital post.
 
 Thanks,
 Paul
State-Changed-From-To: patched->closed 
State-Changed-By: eadler 
State-Changed-When: Fri Mar 15 23:34:45 UTC 2013 
State-Changed-Why:  
MFCed/fixed by now or it will never be MFCed 

http://www.freebsd.org/cgi/query-pr.cgi?pr=120343 
>Unformatted:
