From nobody@FreeBSD.org  Tue Aug 25 17:31:11 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 70053106568B
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 25 Aug 2009 17:31:11 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 5E6528FC26
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 25 Aug 2009 17:31:11 +0000 (UTC)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n7PHVB17094489
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 25 Aug 2009 17:31:11 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n7PHVB4o094488;
	Tue, 25 Aug 2009 17:31:11 GMT
	(envelope-from nobody)
Message-Id: <200908251731.n7PHVB4o094488@www.freebsd.org>
Date: Tue, 25 Aug 2009 17:31:11 GMT
From: Frank Sheiness <syndesis@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: FreeBSD crashing repeatedly in pcpu.h:195
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         138177
>Category:       kern
>Synopsis:       [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:2577
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    cy
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Aug 25 17:40:01 UTC 2009
>Closed-Date:    
>Last-Modified:  Wed Jul 03 05:21:50 UTC 2013
>Originator:     Frank Sheiness
>Release:        7.2-STABLE
>Organization:
>Environment:
FreeBSD ROUTER 7.2-STABLE FreeBSD 7.2-STABLE #0: Mon Aug 24 20:47:37 UTC 2009     root@BUILD-7.0:/usr/obj/usr/src2/src/sys/BUILD  amd64
>Description:
We have FreeBSD servers with this code deployed at several sites.  It is only crashing at one site, but on both servers (original and spare) at that site.  After booting, the server is fine until the interrupts start climbing on the CPU.  Eventually, the server panics.  It can take a few minutes or hours before it crashes.  We suspect some kind of traffic is triggering it.  We blocked everything that has an ether type other than IPv4 or ARP on the switch and it hasn't crashed since (around 12 hours).  Here is the backtrace:

Unread portion of the kernel message buffer:
ge not present
instruction pointer     = 0x8:0xffffffff801b8f68
stack pointer           = 0x10:0xffffff800002d560
frame pointer           = 0x10:0x14cb
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 15 (swi1: net)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 20m47s
Physical memory: 1004 MB
Dumping 1181 MB: 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 718 702 686 670 654 638 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14

#0  doadump () at pcpu.h:195
195             __asm __volatile("movq %%gs:0,%0" : "=r" (td));
(kgdb) where

#0  doadump () at pcpu.h:195
#1  0x0000000000000004 in ?? ()
#2  0xffffffff804067fe in boot (howto=260)
    at /usr/src2/src/sys/kern/kern_shutdown.c:418
#3  0xffffffff80406c12 in panic (fmt=0x104 <Address 0x104 out of bounds>)
    at /usr/src2/src/sys/kern/kern_shutdown.c:574
#4  0xffffffff806a9f73 in trap_fatal (frame=0xffffff0001191390, eva=Variable "eva" is not available.
)
    at /usr/src2/src/sys/amd64/amd64/trap.c:756
#5  0xffffffff806aa345 in trap_pfault (frame=0xffffff800002d4b0, usermode=0)
    at /usr/src2/src/sys/amd64/amd64/trap.c:672
#6  0xffffffff806aac84 in trap (frame=0xffffff800002d4b0)
    at /usr/src2/src/sys/amd64/amd64/trap.c:443
#7  0xffffffff8068f73e in calltrap ()
    at /usr/src2/src/sys/amd64/amd64/exception.S:218
#8  0xffffffff801b8f68 in nat_new (fin=0xffffff800002d700, np=Variable "np" is not available.
)
    at /usr/src2/src/sys/contrib/ipfilter/netinet/ip_nat.c:2577
#9  0xffffffff801bcac0 in fr_checknatout (fin=0xffffff800002d700, 
    passp=0xffffff800002d7ec)
    at /usr/src2/src/sys/contrib/ipfilter/netinet/ip_nat.c:3828
#10 0xffffffff801b02aa in fr_check (ip=0x2, hlen=Variable "hlen" is not available.
)
    at /usr/src2/src/sys/contrib/ipfilter/netinet/fil.c:2624
#11 0xffffffff804d58a1 in pfil_run_hooks (ph=Variable "ph" is not available.
)
    at /usr/src2/src/sys/net/pfil.c:78
#12 0xffffffff8051d64d in ip_output (m=0xffffff00013b6100, opt=Variable "opt" is not available.
)
    at /usr/src2/src/sys/netinet/ip_output.c:443
#13 0xffffffff8051a63d in ip_forward (m=0xffffff00013b6100, srcrt=Variable "srcrt" is not available.
)
    at /usr/src2/src/sys/netinet/ip_input.c:1366
#14 0xffffffff8051be06 in ip_input (m=0xffffff00013b6100)
    at /usr/src2/src/sys/netinet/ip_input.c:609
#15 0xffffffff804c8561 in ether_demux (ifp=0xffffff000c771000, 
    m=0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:834
#16 0xffffffff804c87de in ether_input (ifp=0xffffff000c771000, 
    m=0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:692
#17 0xffffffff804c8486 in ether_demux (ifp=0xffffff00011a9800, 
    m=0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:743
#18 0xffffffff804c87de in ether_input (ifp=0xffffff00011a9800, 
    m=0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:692
#19 0xffffffff80257b02 in em_rxeof (adapter=0xffffff8000211000, count=2)
    at /usr/src2/src/sys/dev/e1000/if_em.c:4539
#20 0xffffffff80259c19 in em_poll (ifp=0xffffff00011a9800, cmd=Variable "cmd" is not available.
)
    at /usr/src2/src/sys/dev/e1000/if_em.c:1594
#21 0xffffffff803fb1ad in netisr_poll ()
    at /usr/src2/src/sys/kern/kern_poll.c:432
#22 0xffffffff804d3c1a in swi_net (dummy=Variable "dummy" is not available.
)
    at /usr/src2/src/sys/net/netisr.c:254
#23 0xffffffff803e22a0 in ithread_loop (arg=0xffffff000117dae0)
    at /usr/src2/src/sys/kern/kern_intr.c:1126
#24 0xffffffff803de94f in fork_exit (
    callout=0xffffffff803e2130 <ithread_loop>, arg=0xffffff000117dae0, 
    frame=0xffffff800002dc80) at /usr/src2/src/sys/kern/kern_fork.c:811
#25 0xffffffff8068fb1e in fork_trampoline ()
    at /usr/src2/src/sys/amd64/amd64/exception.S:554
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000001 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000000000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0x0000000000000000 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000000000000 in ?? ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000000 in ?? ()
#44 0x0000000000000000 in ?? ()
#45 0x0000000000000000 in ?? ()
#46 0x0000000000000000 in ?? ()
#47 0x0000000000000000 in ?? ()
#48 0x0000000000000000 in ?? ()
#49 0x0000000000000000 in ?? ()
#50 0x0000000000af4000 in ?? ()
#51 0xffffffff80981cc0 in tdg_maxid ()
#52 0xffffffff8098e4c0 in tdq_cpu ()
#53 0xffffffff8098e4c0 in tdq_cpu ()
#54 0xffffff0001191390 in ?? ()
#55 0xffffff00011916e0 in ?? ()
#56 0xffffff800002cd08 in ?? ()
#57 0x0000000000000000 in ?? ()
#58 0xffffffff8042e2d8 in sched_switch (td=0xffffffff803e2130, 
    newtd=0x80054b1d0, flags=Variable "flags" is not available.
) at /usr/src2/src/sys/kern/sched_ule.c:1938
#59 0x0000000000000000 in ?? ()
#60 0x0000000000000000 in ?? ()
#61 0x0000000000000000 in ?? ()
#62 0x0000000000000000 in ?? ()
#63 0x0000000000000000 in ?? ()
#64 0x0000000000000000 in ?? ()
#65 0x0000000000000000 in ?? ()
#66 0x0000000000000000 in ?? ()
#67 0x0000000000000000 in ?? ()
#68 0x0000000000000000 in ?? ()
#69 0x0000000000000000 in ?? ()
#70 0x0000000000000000 in ?? ()
#71 0x0000000000000000 in ?? ()
#72 0x0000000000000000 in ?? ()
#73 0x0000000000000000 in ?? ()
#74 0x0000000000000000 in ?? ()
#75 0x0000000000000000 in ?? ()
#76 0x0000000000000000 in ?? ()
#77 0x0000000000000000 in ?? ()
#78 0x0000000000000000 in ?? ()
#79 0x0000000000000000 in ?? ()
#80 0x0000000000000000 in ?? ()
#81 0x0000000000000000 in ?? ()
#82 0x0000000000000000 in ?? ()
#83 0x0000000000000000 in ?? ()
#84 0x0000000000000000 in ?? ()
#85 0x0000000000000000 in ?? ()
#86 0x0000000000000000 in ?? ()
#87 0x0000000000000000 in ?? ()
#88 0x0000000000000000 in ?? ()
#89 0x0000000000000000 in ?? ()
#90 0x0000000000000000 in ?? ()
#91 0x0000000000000000 in ?? ()
#92 0x0000000000000000 in ?? ()
#93 0x0000000000000000 in ?? ()
#94 0x0000000000000000 in ?? ()
#95 0x0000000000000000 in ?? ()
#96 0x0000000000000000 in ?? ()
#97 0x0000000000000000 in ?? ()
#98 0x0000000000000000 in ?? ()
#99 0x0000000000000000 in ?? ()
#100 0x0000000000000000 in ?? ()
#101 0x0000000000000000 in ?? ()
#102 0x0000000000000000 in ?? ()
#103 0x0000000000000000 in ?? ()
#104 0x0000000000000000 in ?? ()
#105 0x0000000000000000 in ?? ()
#106 0x0000000000000000 in ?? ()
#107 0x0000000000000000 in ?? ()
#108 0x0000000000000000 in ?? ()
#109 0x0000000000000000 in ?? ()
#110 0x0000000000000000 in ?? ()
#111 0x0000000000000000 in ?? ()
#112 0x0000000000000000 in ?? ()
#113 0x0000000000000000 in ?? ()
#114 0x0000000000000000 in ?? ()
#115 0x0000000000000000 in ?? ()
#116 0x0000000000000000 in ?? ()
#117 0x0000000000000000 in ?? ()
#118 0x0000000000000000 in ?? ()
#119 0x0000000000000000 in ?? ()
#120 0x0000000000000000 in ?? ()
#121 0x0000000000000000 in ?? ()
#122 0x0000000000000000 in ?? ()
#123 0x0000000000000000 in ?? ()
#124 0x0000000000000000 in ?? ()
#125 0x0000000000000000 in ?? ()
#126 0x0000000000000000 in ?? ()
Cannot access memory at address 0xffffff800002e000
>How-To-Repeat:
Unknown
>Fix:
Unknown

>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback 
State-Changed-By: gavin 
State-Changed-When: Tue Aug 25 18:19:03 UTC 2009 
State-Changed-Why:  
To submitter:  Firstly, can you give me some details about the servers? 
What services do they provide?  You appear to be using the ipfilter 
firewall, what sort of rules are in use? (general info is fine, don't 
need the exact ruleset). 

Also, do you have a core dump from this panic in /var/crash?  If so, can 
you run the following: 
kgdb /boot/kernel/kernel.symbols /var/crash/vmcore.X 

then give the "bt" command to obtain a trace? 

Then, select the "ip_output" frame with (e.g.) "f 12" and give the 
commands "p m" and "p *m". 

Thanks! 


Responsible-Changed-From-To: freebsd-amd64->gavin 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Tue Aug 25 18:19:03 UTC 2009 
Responsible-Changed-Why:  
Track 

http://www.freebsd.org/cgi/query-pr.cgi?pr=138177 

From: Frank Sheiness <syndesis@gmail.com>
To: bug-followup@freebsd.org
Cc: syndesis@gmail.com, gavin@freebsd.org
Subject: Re: kern/138177: [ipfilter] FreeBSD crashing repeatedly in
 ip_nat.c:2577
Date: Tue, 25 Aug 2009 14:32:23 -0500

 The server is a router for about 300 clients.  It provides DNS caching and
 DHCP.  With ipfilter, we use ipnat to do  redirection.  We have sets of
 rules like:
 
 rdr PORT0 XX.XX.XX.XX port 80 -> 172.17.107.254 port 80 tcp
 rdr PORT0 XX.XX.XX.XX port 3074 -> 172.17.107.254 port 3074 tcp
 rdr PORT0 XX.XX.XX.XX port 88 -> 172.17.107.254 port 88 udp
 rdr PORT0 XX.XX.XX.XX port 3074 -> 172.17.107.254 port 3074 udp
 map PORT0 172.17.107.254/32 -> XX.XX.XX.XX portmap tcp/udp 61729:65524
 
 map PORT0 172.17.104.0/32 -> XX.XX.XX.XX portmap tcp/udp 1025:4817
 map PORT0 172.17.104.1/32 -> XX.XX.XX.XX portmap tcp/udp 4818:8610
 map PORT0 172.17.104.2/32 -> XX.XX.XX.XX portmap tcp/udp 8611:12403
 map PORT0 172.17.104.3/32 -> XX.XX.XX.XX portmap tcp/udp 12404:16196
 map PORT0 172.17.104.4/32 -> XX.XX.XX.XX portmap tcp/udp 16197:19989
 map PORT0 172.17.104.5/32 -> XX.XX.XX.XX portmap tcp/udp 19990:23782
 map PORT0 172.17.104.6/32 -> XX.XX.XX.XX portmap tcp/udp 23783:27575
 map PORT0 172.17.104.7/32 -> XX.XX.XX.XX portmap tcp/udp 27576:31368
 map PORT0 172.17.104.8/32 -> XX.XX.XX.XX portmap tcp/udp 31369:35161
 map PORT0 172.17.104.9/32 -> XX.XX.XX.XX portmap tcp/udp 35162:38954
 map PORT0 172.17.104.10/32 -> XX.XX.XX.XX portmap tcp/udp 38955:42747
 map PORT0 172.17.104.11/32 -> XX.XX.XX.XX portmap tcp/udp 42748:46540
 map PORT0 172.17.104.12/32 -> XX.XX.XX.XX portmap tcp/udp 46541:50333
 map PORT0 172.17.104.13/32 -> XX.XX.XX.XX portmap tcp/udp 50334:54126
 map PORT0 172.17.104.14/32 -> XX.XX.XX.XX portmap tcp/udp 54127:57919
 map PORT0 172.17.104.15/32 -> XX.XX.XX.XX portmap tcp/udp 57920:61712
 map PORT0 172.17.104.0/28 -> XX.XX.XX.XX # FOR ICMP
 
 (kgdb) frame 12
 #12 0xffffffff8051d64d in ip_output (m=0xffffff00013b6100, opt=Variable "opt" is not available.
 )
     at /usr/src2/src/sys/netinet/ip_output.c:443
 443             error = pfil_run_hooks(&inet_pfil_hook, &m, ifp, PFIL_OUT, inp);
 (kgdb) p m
 $1 = (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $2 = {m_hdr = {mh_next = 0x0, mh_nextpkt = 0x0, 
     mh_data = 0xffffff000cc9d010 "E", mh_len = 40, mh_flags = 3, mh_type = 1, 
     pad = "\000\000\000\000\000"}, M_dat = {MH = {MH_pkthdr = {
         rcvif = 0xffffff000c771000, header = 0x0, len = 40, csum_flags = 768, 
         csum_data = 0, tso_segsz = 0, ether_vtag = 91, tags = {
           slh_first = 0x0}}, MH_dat = {MH_ext = {
           ext_buf = 0xffffff000cc9d000 "\002", ext_free = 0, ext_args = 0x0, 
           ext_size = 2048, ref_cnt = 0xffffff000c9c79b4, ext_type = 6}, 
         MH_databuf = "\000\f\000", '\0' <repeats 17 times>, "\b\000\000*\002Ky\234\f\000\006\000\000\000eB\r[\217r`?\205B=\0202!\234vw\214L\003\220\027\217C6\226x&\r\217=\005? \225\206]cW\225\vj\025\202cr\210\\Y7\235\206\016t(0\213\233\232V\006\203z\021%hwt\037EowsR~tNd|\003Z?>u\230#, \017"}}, 
     M_databuf = "\000\020w\f\000\000\000\000\000\000\000\000\000(\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\000\000\000\000\000\f\000", '\0' <repeats 17 times>, "\b\000\000*\002Ky\234\f\000\006\000\000\000eB\r[\217r`?\205B=\0202!\234vw\214L\003\220\027\217C6\226x&\r\217=\005? \225\206]cW\225\vj\025\202cr\210\\Y7\235\206\016t(0\213\233\232V\006\203z\021%hwt\037EowsR~tNd|\003Z?>u\230"...}}
 

From: Gavin Atkinson <gavin@FreeBSD.org>
To: Frank Sheiness <frank@korcett.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/138177: [ipfilter] FreeBSD crashing repeatedly in ip_nat.c:2577
Date: Mon, 31 Aug 2009 11:53:15 +0100 (BST)

 Hi,
 
 On Wed, 26 Aug 2009, Frank Sheiness wrote:
 > Just wanted to let you know what we tried switching to 4BSD from ULE and
 > it still crashed (same place).  Have you had a chance to determine a cause
 > or fix?  The server at that site is crashing a lot so anything you can tell
 > us about maybe blocking certain traffic would be helpful.
 
 Sorry, I haven't managed to determine what's happening here yet.  I've 
 sent an email to the ipfilter maintainer as it looks likely the problem is 
 related to NATting being done within ipfilter, but so far I've not heard 
 any response.  I'll wait a couple more days and then try to get somebody 
 else interested.
 
 In the mean time, it would be useful if you could find out more 
 information from your crash dump in the same way you did last time: 
 select each frame in turn from the nat_new frame down to the em_rxeof 
 frame, and print all of the variables passed in to functions (with p *m) 
 as well as any other variables that look like they might be of interest.
 
 Lastly, there seems to be some oddity with the backtrace:
 
 #8 0xffffffff801b8f68 in nat_new (fin=0xffffff800002d700, np=Variable "np" 
 is not available.
 ) at /usr/src2/src/sys/contrib/ipfilter/netinet/ip_nat.c:2577
 
 However, line 2577 doesn't appear to be within the nat_new function.
 
 Is it possible that you have local modifications to the code?  Or your 
 kernel isn't actually 7.2-STABLE?  What version of 
 src/sys/contrib/ipfilter/netinet/ip_nat.c are you using?
 
 Thanks,
 
 Gavin

From: Frank Sheiness <syndesis@gmail.com>
To: Gavin Atkinson <gavin@FreeBSD.org>
Cc: Frank Sheiness <syndesis@gmail.com>, bug-followup@FreeBSD.org
Subject: Re: kern/138177: [ipfilter] FreeBSD crashing repeatedly in
 ip_nat.c:2577
Date: Mon, 31 Aug 2009 18:39:23 -0500

 --LZvS9be/3tNcYl/X
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 
 Hello,
 
 Thank you for the update.  Attaching some more debugging output.  Please
 let me know if anything else would be helpful.
 
 As far as ip_nat.c:2577 I don't know why that is showing the incorrect
 function name.  It's version v 1.42.2.1 2007/10/31.  That build was
 7.2-STABLE with no local modifications.  It was built a week or two ago
 with the latest sources.  Looks like that file hasn't changed in this
 branch since 2007.
 
 On Mon, Aug 31, 2009 at 11:53:15AM +0100, Gavin Atkinson wrote:
 > Hi,
 > 
 > On Wed, 26 Aug 2009, Frank Sheiness wrote:
 > > Just wanted to let you know what we tried switching to 4BSD from ULE and
 > > it still crashed (same place).  Have you had a chance to determine a cause
 > > or fix?  The server at that site is crashing a lot so anything you can tell
 > > us about maybe blocking certain traffic would be helpful.
 > 
 > Sorry, I haven't managed to determine what's happening here yet.  I've 
 > sent an email to the ipfilter maintainer as it looks likely the problem is 
 > related to NATting being done within ipfilter, but so far I've not heard 
 > any response.  I'll wait a couple more days and then try to get somebody 
 > else interested.
 > 
 > In the mean time, it would be useful if you could find out more 
 > information from your crash dump in the same way you did last time: 
 > select each frame in turn from the nat_new frame down to the em_rxeof 
 > frame, and print all of the variables passed in to functions (with p *m) 
 > as well as any other variables that look like they might be of interest.
 > 
 > Lastly, there seems to be some oddity with the backtrace:
 > 
 > #8 0xffffffff801b8f68 in nat_new (fin=0xffffff800002d700, np=Variable "np" 
 > is not available.
 > ) at /usr/src2/src/sys/contrib/ipfilter/netinet/ip_nat.c:2577
 > 
 > However, line 2577 doesn't appear to be within the nat_new function.
 > 
 > Is it possible that you have local modifications to the code?  Or your 
 > kernel isn't actually 7.2-STABLE?  What version of 
 > src/sys/contrib/ipfilter/netinet/ip_nat.c are you using?
 > 
 > Thanks,
 > 
 > Gavin
 
 --LZvS9be/3tNcYl/X
 Content-Type: text/plain; charset=unknown-8bit
 Content-Disposition: attachment; filename="script.out"
 Content-Transfer-Encoding: quoted-printable
 
 Script started on Mon Aug 31 16:19:24 2009
 You have mail.
 BUILD-7# kgdb kernel.debug /usr/tmp/2/vmcore.0=20
 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:=
  Undefined symbol "ps_pglobal_lookup"]
 GNU gdb 6.1.1 [FreeBSD]
 Copyright 2004 Free Software Foundation, Inc.
 GDB is free software, covered by the GNU General Public License, and you are
 welcome to change it and/or distribute copies of it under certain condition=
 s.
 Type "show copying" to see the conditions.
 There is absolutely no warranty for GDB.  Type "show warranty" for details.
 This GDB was configured as "amd64-marcel-freebsd".
 
 Unread portion of the kernel message buffer:
 ge not present
 instruction pointer	=3D 0x8:0xffffffff801b8f68
 stack pointer	        =3D 0x10:0xffffff800002d560
 frame pointer	        =3D 0x10:0x14cb
 code segment		=3D base 0x0, limit 0xfffff, type 0x1b
 			=3D DPL 0, pres 1, long 1, def32 0, gran 1
 processor eflags	=3D interrupt enabled, resume, IOPL =3D 0
 current process		=3D 15 (swi1: net)
 trap number		=3D 12
 panic: page fault
 cpuid =3D 0
 Uptime: 20m47s
 Physical memory: 1004 MB
 Dumping 1181 MB: 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 1006 990=
  974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 718 702 68=
 6 670 654 638 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 3=
 82 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 7=
 8 62 46 30 14
 
 #0  doadump () at pcpu.h:195
 195		__asm __volatile("movq %%gs:0,%0" : "=3Dr" (td));
 (kgdb) f 8
 #8  0xffffffff801b8f68 in nat_new (fin=3D0xffffff800002d700, np=3DVariable =
 "np" is not available.
 )
     at /usr/src2/src/sys/contrib/ipfilter/netinet/ip_nat.c:2577
 2577			nat->nat_seqnext[0] =3D ntohl(tcp->th_seq);
 (kgdb) p fin
 $1 =3D (fr_info_t *) 0xffffff800002d700
 (kgdb) p *fin
 $2 =3D {fin_ifp =3D 0xffffff000116b800, fin_fi =3D {fi_v =3D 4, fi_xx =3D 0=
 , fi_tos =3D 0,=20
     fi_ttl =3D 253, fi_p =3D 6, fi_optmsk =3D 0, fi_src =3D {i6 =3D {311060=
 5228, 0, 0,=20
         0}, in4 =3D {s_addr =3D 3110605228}, in6 =3D {__u6_addr =3D {
           __u6_addr8 =3D "=AC\021h=B9", '\0' <repeats 11 times>, __u6_addr1=
 6 =3D {
             4524, 47464, 0, 0, 0, 0, 0, 0}, __u6_addr32 =3D {3110605228, 0,=
  0,=20
             0}}}, vptr =3D {0xb96811ac, 0x0}, lptr =3D {0xb96811ac, 0}, i6u=
 n =3D {
         type =3D 4524, subtype =3D 47464, label =3D '\0' <repeats 11 times>=
 }},=20
     fi_dst =3D {i6 =3D {1680702794, 0, 0, 0}, in4 =3D {s_addr =3D 168070279=
 4}, in6 =3D {
         __u6_addr =3D {__u6_addr8 =3D "J}-d", '\0' <repeats 11 times>,=20
           __u6_addr16 =3D {32074, 25645, 0, 0, 0, 0, 0, 0}, __u6_addr32 =3D=
  {
             1680702794, 0, 0, 0}}}, vptr =3D {0x642d7d4a, 0x0}, lptr =3D {
         0x642d7d4a, 0}, i6un =3D {type =3D 32074, subtype =3D 25645,=20
         label =3D '\0' <repeats 11 times>}}, fi_secmsk =3D 0, fi_auth =3D 0=
 ,=20
     fi_flx =3D 8205, fi_tcpmsk =3D 0, fi_res1 =3D 0}, fin_dat =3D {fid_16 =
 =3D {0, 0},=20
     fid_32 =3D 0}, fin_out =3D 1, fin_rev =3D 0, fin_hlen =3D 20, fin_tcpf =
 =3D 0 '\0',=20
   fin_icode =3D 0 '\0', fin_rule =3D 4294967295,=20
   fin_group =3D "=FF", '\0' <repeats 14 times>, fin_fr =3D 0x0,=20
   fin_dp =3D 0xffffff000cc9d024, fin_dlen =3D 20, fin_plen =3D 40, fin_ipof=
 f =3D 0,=20
   fin_id =3D 0, fin_off =3D 8, fin_depth =3D 0, fin_error =3D 51, fin_cksum=
  =3D 0,=20
   fin_nat =3D 0x0, fin_state =3D 0x0, fin_nattag =3D 0x0, fin_exthdr =3D 0x=
 0,=20
   fin_ip =3D 0xffffff000cc9d010, fin_mp =3D 0xffffff800002d850,=20
   fin_m =3D 0xffffff00013b6100}
 (kgdb) p natsave
 $3 =3D (nat_t **) 0x0
 (kgdb) p flags
 $4 =3D 0
 (kgdb) p direction
 $5 =3D 1
 (kgdb) p nat_rules
 $6 =3D (ipnat_t **) 0xffffff8000561000
 (kgdb) p *nat_rules
 $7 =3D (ipnat_t *) 0x0
 (kgdb) f 9
 #9  0xffffffff801bcac0 in fr_checknatout (fin=3D0xffffff800002d700,=20
     passp=3D0xffffff800002d7ec)
     at /usr/src2/src/sys/contrib/ipfilter/netinet/ip_nat.c:3828
 3828				if ((nat =3D nat_new(fin, np, NULL, nflags,
 (kgdb) p fin
 $8 =3D (fr_info_t *) 0xffffff800002d700
 (kgdb) p *fin
 $9 =3D {fin_ifp =3D 0xffffff000116b800, fin_fi =3D {fi_v =3D 4, fi_xx =3D 0=
 , fi_tos =3D 0,=20
     fi_ttl =3D 253, fi_p =3D 6, fi_optmsk =3D 0, fi_src =3D {i6 =3D {311060=
 5228, 0, 0,=20
         0}, in4 =3D {s_addr =3D 3110605228}, in6 =3D {__u6_addr =3D {
           __u6_addr8 =3D "=AC\021h=B9", '\0' <repeats 11 times>, __u6_addr1=
 6 =3D {
             4524, 47464, 0, 0, 0, 0, 0, 0}, __u6_addr32 =3D {3110605228, 0,=
  0,=20
             0}}}, vptr =3D {0xb96811ac, 0x0}, lptr =3D {0xb96811ac, 0}, i6u=
 n =3D {
         type =3D 4524, subtype =3D 47464, label =3D '\0' <repeats 11 times>=
 }},=20
     fi_dst =3D {i6 =3D {1680702794, 0, 0, 0}, in4 =3D {s_addr =3D 168070279=
 4}, in6 =3D {
         __u6_addr =3D {__u6_addr8 =3D "J}-d", '\0' <repeats 11 times>,=20
           __u6_addr16 =3D {32074, 25645, 0, 0, 0, 0, 0, 0}, __u6_addr32 =3D=
  {
             1680702794, 0, 0, 0}}}, vptr =3D {0x642d7d4a, 0x0}, lptr =3D {
         0x642d7d4a, 0}, i6un =3D {type =3D 32074, subtype =3D 25645,=20
         label =3D '\0' <repeats 11 times>}}, fi_secmsk =3D 0, fi_auth =3D 0=
 ,=20
     fi_flx =3D 8205, fi_tcpmsk =3D 0, fi_res1 =3D 0}, fin_dat =3D {fid_16 =
 =3D {0, 0},=20
     fid_32 =3D 0}, fin_out =3D 1, fin_rev =3D 0, fin_hlen =3D 20, fin_tcpf =
 =3D 0 '\0',=20
   fin_icode =3D 0 '\0', fin_rule =3D 4294967295,=20
   fin_group =3D "=FF", '\0' <repeats 14 times>, fin_fr =3D 0x0,=20
   fin_dp =3D 0xffffff000cc9d024, fin_dlen =3D 20, fin_plen =3D 40, fin_ipof=
 f =3D 0,=20
   fin_id =3D 0, fin_off =3D 8, fin_depth =3D 0, fin_error =3D 51, fin_cksum=
  =3D 0,=20
   fin_nat =3D 0x0, fin_state =3D 0x0, fin_nattag =3D 0x0, fin_exthdr =3D 0x=
 0,=20
   fin_ip =3D 0xffffff000cc9d010, fin_mp =3D 0xffffff800002d850,=20
   fin_m =3D 0xffffff00013b6100}
 (kgdb) p passp
 $10 =3D (u_32_t *) 0xffffff800002d7ec
 (kgdb) p *passp
 $11 =3D 134217730
 (kgdb) f 10
 #10 0xffffffff801b02aa in fr_check (ip=3D0x2, hlen=3DVariable "hlen" is not=
  available.
 )
     at /usr/src2/src/sys/contrib/ipfilter/netinet/fil.c:2624
 2624			if (fr_checknatout(fin, &pass) =3D=3D -1) {
 (kgdb) p ip
 $12 =3D (ip_t *) 0x2
 (kgdb) p hlen
 Variable "hlen" is not available.
 (kgdb) p ifp
 Variable "ifp" is not available.
 (kgdb) p out
 $13 =3D 1
 (kgdb) p pass
 $14 =3D 134217730
 (kgdb) f 11
 #11 0xffffffff804d58a1 in pfil_run_hooks (ph=3DVariable "ph" is not availab=
 le.
 )
     at /usr/src2/src/sys/net/pfil.c:78
 78				rv =3D (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, inp);
 (kgdb) p mp
 $15 =3D (struct mbuf **) 0xffffff800002d8f8
 (kgdb) p *mp
 $16 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p **mp
 $17 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p dir
 $18 =3D 2
 (kgdb) p inp
 $19 =3D (struct inpcb *) 0x0
 (kgdb) f 12
 #12 0xffffffff8051d64d in ip_output (m=3D0xffffff00013b6100, opt=3DVariable=
  "opt" is not available.
 )
     at /usr/src2/src/sys/netinet/ip_output.c:443
 443		error =3D pfil_run_hooks(&inet_pfil_hook, &m, ifp, PFIL_OUT, inp);
 (kgdb) p m
 $20 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $21 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ro
 $22 =3D (struct route *) 0xffffff800002d970
 (kgdb) p *ro
 $23 =3D {ro_rt =3D 0xffffff000c5df6c8, ro_dst =3D {sa_len =3D 16 '\020',=20
     sa_family =3D 2 '\002',=20
     sa_data =3D "\000\000J}-d\000\000\000\000\000\000\000"}}
 (kgdb) p flags
 $24 =3D 1
 (kgdb) p imo
 $25 =3D (struct ip_moptions *) 0x0
 (kgdb) p inp
 $26 =3D (struct inpcb *) 0x0
 (kgdb) p error
 $27 =3D 0
 (kgdb) f 13
 #13 0xffffffff8051a63d in ip_forward (m=3D0xffffff00013b6100, srcrt=3DVaria=
 ble "srcrt" is not available.
 )
     at /usr/src2/src/sys/netinet/ip_input.c:1366
 1366		error =3D ip_output(m, NULL, &ro, IP_FORWARDING, NULL, NULL);
 (kgdb) p m
 $28 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $29 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ro
 $30 =3D {ro_rt =3D 0xffffff000c5df6c8, ro_dst =3D {sa_len =3D 16 '\020',=20
     sa_family =3D 2 '\002',=20
     sa_data =3D "\000\000J}-d\000\000\000\000\000\000\000"}}
 (kgdb) f 14
 #14 0xffffffff8051be06 in ip_input (m=3D0xffffff00013b6100)
     at /usr/src2/src/sys/netinet/ip_input.c:609
 609			ip_forward(m, dchg);
 (kgdb) p m
 $31 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $32 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p dchg
 Variable "dchg" is not available.
 (kgdb) f 15
 #15 0xffffffff804c8561 in ether_demux (ifp=3D0xffffff000c771000,=20
     m=3D0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:834
 834		netisr_dispatch(isr, m);
 (kgdb) p m
 $33 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $34 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ifp
 $35 =3D (struct ifnet *) 0xffffff000c771000
 (kgdb) p *ifp
 $36 =3D {if_softc =3D 0xffffff000c739600, if_l2com =3D 0xffffff000c71e220,=
 =20
   if_link =3D {tqe_next =3D 0xffffff000c770800, tqe_prev =3D 0xffffff000c77=
 1810},=20
   if_xname =3D "bridge0\000\000\000\000\000\000\000\000",=20
   if_dname =3D 0xffffffff80703f54 "bridge", if_dunit =3D 0, if_addrhead =3D=
  {
     tqh_first =3D 0xffffff000c739000, tqh_last =3D 0xffffff000c91ccb8},=20
   if_klist =3D {kl_list =3D {slh_first =3D 0x0},=20
     kl_lock =3D 0xffffffff803d8b40 <knlist_mtx_lock>,=20
     kl_unlock =3D 0xffffffff803d8590 <knlist_mtx_unlock>,=20
     kl_locked =3D 0xffffffff803d8570 <knlist_mtx_locked>,=20
     kl_lockarg =3D 0xffffffff809772e0}, if_pcount =3D 0, if_carp =3D 0x0,=
 =20
   if_bpf =3D 0xffffff000c6cc980, if_index =3D 257, if_timer =3D 0,=20
   if_vlantrunk =3D 0x0, if_flags =3D 2131971, if_capabilities =3D 0,=20
   if_capenable =3D 0, if_linkmib =3D 0x0, if_linkmiblen =3D 0, if_data =3D {
     ifi_type =3D 209 '=D1', ifi_physical =3D 0 '\0', ifi_addrlen =3D 6 '\00=
 6',=20
     ifi_hdrlen =3D 14 '\016', ifi_link_state =3D 0 '\0', ifi_spare_char1 =
 =3D 0 '\0',=20
     ifi_spare_char2 =3D 0 '\0', ifi_datalen =3D 152 '\230', ifi_mtu =3D 150=
 0,=20
     ifi_metric =3D 0, ifi_baudrate =3D 0, ifi_ipackets =3D 2439128, ifi_ier=
 rors =3D 0,=20
     ifi_opackets =3D 3951021, ifi_oerrors =3D 0, ifi_collisions =3D 0,=20
     ifi_ibytes =3D 12413395, ifi_obytes =3D 10464551014, ifi_imcasts =3D 74=
 266,=20
     ifi_omcasts =3D 16, ifi_iqdrops =3D 0, ifi_noproto =3D 0, ifi_hwassist =
 =3D 0,=20
     ifi_epoch =3D 4, ifi_lastchange =3D {tv_sec =3D 1251173695, tv_usec =3D=
  190460}},=20
   if_multiaddrs =3D {tqh_first =3D 0xffffff000c761a40,=20
     tqh_last =3D 0xffffff000c674c40}, if_amcount =3D 0,=20
   if_output =3D 0xffffffff804c7f40 <ether_output>,=20
   if_input =3D 0xffffffff804c8610 <ether_input>,=20
   if_start =3D 0xffffffff804c4690 <bridge_start>,=20
   if_ioctl =3D 0xffffffff804c32f0 <bridge_ioctl>, if_watchdog =3D 0,=20
   if_init =3D 0xffffffff804c2e00 <bridge_init>,=20
   if_resolvemulti =3D 0xffffffff804c72a0 <ether_resolvemulti>,=20
   if_addr =3D 0xffffff000c739000, if_llsoftc =3D 0x0, if_drv_flags =3D 64,=
 =20
   if_spare_flags2 =3D 0, if_snd =3D {ifq_head =3D 0x0, ifq_tail =3D 0x0, if=
 q_len =3D 0,=20
     ifq_maxlen =3D 50, ifq_drops =3D 0, ifq_mtx =3D {lock_object =3D {
         lo_name =3D 0xffffff000c771020 "bridge0",=20
         lo_type =3D 0xffffffff80743980 "if send queue", lo_flags =3D 169738=
 24,=20
         lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
       mtx_lock =3D 4, mtx_recurse =3D 0}, ifq_drv_head =3D 0x0, ifq_drv_tai=
 l =3D 0x0,=20
     ifq_drv_len =3D 0, ifq_drv_maxlen =3D 50, altq_type =3D 0, altq_flags =
 =3D 1,=20
     altq_disc =3D 0x0, altq_ifp =3D 0xffffff000c771000, altq_enqueue =3D 0,=
 =20
     altq_dequeue =3D 0, altq_request =3D 0, altq_clfier =3D 0x0, altq_class=
 ify =3D 0,=20
     altq_tbr =3D 0x0, altq_cdnr =3D 0x0},=20
   if_broadcastaddr =3D 0xffffffff80744260 "=FF=FF=FF=FF=FF=FF"p *ifp
 , if_bridge =3D 0x0,=20
   lltables =3D 0x0, if_label =3D 0x0, if_prefixhead =3D {tqh_first =3D 0x0,=
 =20
     tqh_last =3D 0xffffff000c771298}, if_afdata =3D {0x0 <repeats 28 times>=
 ,=20
     0xffffff000c70e900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},=20
   if_afdata_initialized =3D 2, if_afdata_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff80743976 "if_afdata",=20
       lo_type =3D 0xffffffff80743976 "if_afdata", lo_flags =3D 16973824,=20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_starttask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804ba8f0 <if_start_deferred>,=20
     ta_context =3D 0xffffff000c771000}, if_linktask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804be2b0 <do_link_state_change>,=20
     ta_context =3D 0xffffff000c771000}, if_addr_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff8074396a "if_addr_mtx",=20
       lo_type =3D 0xffffffff8074396a "if_addr_mtx", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_clones =3D {le_next =3D 0x0,=20
     le_prev =3D 0xffffffff80921c70}, if_groups =3D {
     tqh_first =3D 0xffffff000c70e9a0, tqh_last =3D 0xffffff000c70e8c8},=20
   if_pf_kif =3D 0x0, if_lagg =3D 0x0, if_pspare =3D {0x0, 0x0, 0x0, 0x0, 0x=
 0, 0x0,=20
     0x0, 0x0, 0x0, 0x0}, if_ispare =3D {0, 0}}
 (kgdb) f 16
 #16 0xffffffff804c87de in ether_input (ifp=3D0xffffff000c771000,=20
     m=3D0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:692
 692		ether_demux(ifp, m);
 (kgdb) p m
 $37 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $38 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ifp
 $39 =3D (struct ifnet *) 0xffffff000c771000
 (kgdb) p *ifp
 $40 =3D {if_softc =3D 0xffffff000c739600, if_l2com =3D 0xffffff000c71e220,=
 =20
   if_link =3D {tqe_next =3D 0xffffff000c770800, tqe_prev =3D 0xffffff000c77=
 1810},=20
   if_xname =3D "bridge0\000\000\000\000\000\000\000\000",=20
   if_dname =3D 0xffffffff80703f54 "bridge", if_dunit =3D 0, if_addrhead =3D=
  {
     tqh_first =3D 0xffffff000c739000, tqh_last =3D 0xffffff000c91ccb8},=20
   if_klist =3D {kl_list =3D {slh_first =3D 0x0},=20
     kl_lock =3D 0xffffffff803d8b40 <knlist_mtx_lock>,=20
     kl_unlock =3D 0xffffffff803d8590 <knlist_mtx_unlock>,=20
     kl_locked =3D 0xffffffff803d8570 <knlist_mtx_locked>,=20
     kl_lockarg =3D 0xffffffff809772e0}, if_pcount =3D 0, if_carp =3D 0x0,=
 =20
   if_bpf =3D 0xffffff000c6cc980, if_index =3D 257, if_timer =3D 0,=20
   if_vlantrunk =3D 0x0, if_flags =3D 2131971, if_capabilities =3D 0,=20
   if_capenable =3D 0, if_linkmib =3D 0x0, if_linkmiblen =3D 0, if_data =3D {
     ifi_type =3D 209 '=D1', ifi_physical =3D 0 '\0', ifi_addrlen =3D 6 '\00=
 6',=20
     ifi_hdrlen =3D 14 '\016', ifi_link_state =3D 0 '\0', ifi_spare_char1 =
 =3D 0 '\0',=20
     ifi_spare_char2 =3D 0 '\0', ifi_datalen =3D 152 '\230', ifi_mtu =3D 150=
 0,=20
     ifi_metric =3D 0, ifi_baudrate =3D 0, ifi_ipackets =3D 2439128, ifi_ier=
 rors =3D 0,=20
     ifi_opackets =3D 3951021, ifi_oerrors =3D 0, ifi_collisions =3D 0,=20
     ifi_ibytes =3D 12413395, ifi_obytes =3D 10464551014, ifi_imcasts =3D 74=
 266,=20
     ifi_omcasts =3D 16, ifi_iqdrops =3D 0, ifi_noproto =3D 0, ifi_hwassist =
 =3D 0,=20
     ifi_epoch =3D 4, ifi_lastchange =3D {tv_sec =3D 1251173695, tv_usec =3D=
  190460}},=20
   if_multiaddrs =3D {tqh_first =3D 0xffffff000c761a40,=20
     tqh_last =3D 0xffffff000c674c40}, if_amcount =3D 0,=20
   if_output =3D 0xffffffff804c7f40 <ether_output>,=20
   if_input =3D 0xffffffff804c8610 <ether_input>,=20
   if_start =3D 0xffffffff804c4690 <bridge_start>,=20
   if_ioctl =3D 0xffffffff804c32f0 <bridge_ioctl>, if_watchdog =3D 0,=20
   if_init =3D 0xffffffff804c2e00 <bridge_init>,=20
   if_resolvemulti =3D 0xffffffff804c72a0 <ether_resolvemulti>,=20
   if_addr =3D 0xffffff000c739000, if_llsoftc =3D 0x0, if_drv_flags =3D 64,=
 =20
   if_spare_flags2 =3D 0, if_snd =3D {ifq_head =3D 0x0, ifq_tail =3D 0x0, if=
 q_len =3D 0,=20
     ifq_maxlen =3D 50, ifq_drops =3D 0, ifq_mtx =3D {lock_object =3D {
         lo_name =3D 0xffffff000c771020 "bridge0",=20
         lo_type =3D 0xffffffff80743980 "if send queue", lo_flags =3D 169738=
 24,=20
         lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
       mtx_lock =3D 4, mtx_recurse =3D 0}, ifq_drv_head =3D 0x0, ifq_drv_tai=
 l =3D 0x0,=20
     ifq_drv_len =3D 0, ifq_drv_maxlen =3D 50, altq_type =3D 0, altq_flags =
 =3D 1,=20
     altq_disc =3D 0x0, altq_ifp =3D 0xffffff000c771000, altq_enqueue =3D 0,=
 =20
     altq_dequeue =3D 0, altq_request =3D 0, altq_clfier =3D 0x0, altq_class=
 ify =3D 0,=20
     altq_tbr =3D 0x0, altq_cdnr =3D 0x0},=20
   if_broadcastaddr =3D 0xffffffff80744260 "=FF=FF=FF=FF=FF=FF", if_bridge =
 =3D 0x0,=20
   lltables =3D 0x0, if_label =3D 0x0, if_prefixhead =3D {tqh_first =3D 0x0,=
 =20
     tqh_last =3D 0xffffff000c771298}, if_afdata =3D {0x0 <repeats 28 times>=
 ,=20
     0xffffff000c70e900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},=20
   if_afdata_initialized =3D 2, if_afdata_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff80743976 "if_afdata",=20
       lo_type =3D 0xffffffff80743976 "if_afdata", lo_flags =3D 16973824,=20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_starttask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804ba8f0 <if_start_deferred>,=20
     ta_context =3D 0xffffff000c771000}, if_linktask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804be2b0 <do_link_state_change>,=20
     ta_context =3D 0xffffff000c771000}, if_addr_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff8074396a "if_addr_mtx",=20
       lo_type =3D 0xffffffff8074396a "if_addr_mtx", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_clones =3D {le_next =3D 0x0,=20
     le_prev =3D 0xffffffff80921c70}, if_groups =3D {
     tqh_first =3D 0xffffff000c70e9a0, tqh_last =3D 0xffffff000c70e8c8},=20
   if_pf_kif =3D 0x0, if_lagg =3D 0x0, if_pspare =3D {0x0, 0x0, 0x0, 0x0, 0x=
 0, 0x0,=20
     0x0, 0x0, 0x0, 0x0}, if_ispare =3D {0, 0}}
 (kgdb) f 17
 #17 0xffffffff804c8486 in ether_demux (ifp=3D0xffffff00011a9800,=20
     m=3D0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:743
 743			(*vlan_input_p)(ifp, m);
 (kgdb) p vlan_input_p
 $41 =3D (void (*)(struct ifnet *, struct mbuf *)) 0xffffffff804d2a20 <vlan_=
 input>
 (kgdb) p *vlan_input_p
 $42 =3D {void (struct ifnet *, struct mbuf *)} 0xffffffff804d2a20 <vlan_inp=
 ut>
 (kgdb) p m
 $43 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $44 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ifp
 $45 =3D (struct ifnet *) 0xffffff00011a9800
 (kgdb) p *ifp
 $46 =3D {if_softc =3D 0xffffff8000211000, if_l2com =3D 0xffffff00012d97c0,=
 =20
   if_link =3D {tqe_next =3D 0xffffff00013c1000, tqe_prev =3D 0xffffff000116=
 b810},=20
   if_xname =3D "PORT1\000\000\000\000\000\000\000\000\000\000",=20
   if_dname =3D 0xffffff000117c9d8 "em", if_dunit =3D 1, if_addrhead =3D {
     tqh_first =3D 0xffffff00012e5800, tqh_last =3D 0xffffff00012e58b8},=20
   if_klist =3D {kl_list =3D {slh_first =3D 0x0},=20
     kl_lock =3D 0xffffffff803d8b40 <knlist_mtx_lock>,=20
     kl_unlock =3D 0xffffffff803d8590 <knlist_mtx_unlock>,=20
     kl_locked =3D 0xffffffff803d8570 <knlist_mtx_locked>,=20
     kl_lockarg =3D 0xffffffff809772e0}, if_pcount =3D 250, if_carp =3D 0x0,=
 =20
   if_bpf =3D 0xffffff00012ddd00, if_index =3D 2, if_timer =3D 0,=20
   if_vlantrunk =3D 0xffffff000c6cca00, if_flags =3D 35075, if_capabilities =
 =3D 475,=20
   if_capenable =3D 475, if_linkmib =3D 0x0, if_linkmiblen =3D 0, if_data =
 =3D {
     ifi_type =3D 6 '\006', ifi_physical =3D 0 '\0', ifi_addrlen =3D 6 '\006=
 ',=20
     ifi_hdrlen =3D 18 '\022', ifi_link_state =3D 2 '\002',=20
     ifi_spare_char1 =3D 0 '\0', ifi_spare_char2 =3D 0 '\0',=20
     ifi_datalen =3D 152 '\230', ifi_mtu =3D 1500, ifi_metric =3D 0,=20
     ifi_baudrate =3D 1000000000, ifi_ipackets =3D 2441940, ifi_ierrors =3D =
 677,=20
     ifi_opackets =3D 3951735, ifi_oerrors =3D 0, ifi_collisions =3D 0,=20
     ifi_ibytes =3D 385014266, ifi_obytes =3D 5239057455, ifi_imcasts =3D 38=
 496,=20
     ifi_omcasts =3D 16, ifi_iqdrops =3D 0, ifi_noproto =3D 4, ifi_hwassist =
 =3D 38,=20
     ifi_epoch =3D 1, ifi_lastchange =3D {tv_sec =3D 1251173561, tv_usec =3D=
  705068}},=20
   if_multiaddrs =3D {tqh_first =3D 0xffffff000c91bb40,=20
     tqh_last =3D 0xffffff000c91bb40}, if_amcount =3D 0,=20
   if_output =3D 0xffffffff804c7f40 <ether_output>,=20
   if_input =3D 0xffffffff804c8610 <ether_input>,=20
   if_start =3D 0xffffffff802570e0 <em_start>,=20
   if_ioctl =3D 0xffffffff80258de0 <em_ioctl>, if_watchdog =3D 0,=20
   if_init =3D 0xffffffff80255d40 <em_init>,=20
   if_resolvemulti =3D 0xffffffff804c72a0 <ether_resolvemulti>,=20
   if_addr =3D 0xffffff00012e5800, if_llsoftc =3D 0x0, if_drv_flags =3D 64,=
 =20
   if_spare_flags2 =3D 0, if_snd =3D {ifq_head =3D 0x0, ifq_tail =3D 0x0, if=
 q_len =3D 0,=20
     ifq_maxlen =3D 255, ifq_drops =3D 0, ifq_mtx =3D {lock_object =3D {
         lo_name =3D 0xffffff00011a9820 "PORT1",=20
         lo_type =3D 0xffffffff80743980 "if send queue", lo_flags =3D 169738=
 24,=20
         lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
       mtx_lock =3D 4, mtx_recurse =3D 0}, ifq_drv_head =3D 0x0, ifq_drv_tai=
 l =3D 0x0,=20
     ifq_drv_len =3D 0, ifq_drv_maxlen =3D 255, altq_type =3D 0, altq_flags =
 =3D 1,=20
     altq_disc =3D 0x0, altq_ifp =3D 0xffffff00011a9800, altq_enqueue =3D 0,=
 =20
     altq_dequeue =3D 0, altq_request =3D 0, altq_clfier =3D 0x0, altq_class=
 ify =3D 0,=20
     altq_tbr =3D 0x0, altq_cdnr =3D 0x0},=20
   if_broadcastaddr =3D 0xffffffff80744260 "=FF=FF=FF=FF=FF=FF", if_bridge =
 =3D 0x0,=20
   lltables =3D 0x0, if_label =3D 0x0, if_prefixhead =3D {tqh_first =3D 0x0,=
 =20
     tqh_last =3D 0xffffff00011a9a98}, if_afdata =3D {0x0 <repeats 28 times>=
 ,=20
     0xffffff000136d2e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},=20
   if_afdata_initialized =3D 2, if_afdata_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff80743976 "if_afdata",=20
       lo_type =3D 0xffffffff80743976 "if_afdata", lo_flags =3D 16973824,=20
       lo_witness_data =3D {lod_list =3D {p *ifp
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_starttask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804ba8f0 <if_start_deferred>,=20
     ta_context =3D 0xffffff00011a9800}, if_linktask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804be2b0 <do_link_state_change>,=20
     ta_context =3D 0xffffff00011a9800}, if_addr_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff8074396a "if_addr_mtx",=20
       lo_type =3D 0xffffffff8074396a "if_addr_mtx", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_clones =3D {le_next =3D 0x0,=20
     le_prev =3D 0x0}, if_groups =3D {tqh_first =3D 0xffffff00012dbb80,=20
     tqh_last =3D 0xffffff00012dbb88}, if_pf_kif =3D 0x0, if_lagg =3D 0x0,=
 =20
   if_pspare =3D {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_ispa=
 re =3D {
     0, 0}}
 (kgdb) f 18
 #18 0xffffffff804c87de in ether_input (ifp=3D0xffffff00011a9800,=20
     m=3D0xffffff00013b6100) at /usr/src2/src/sys/net/if_ethersubr.c:692
 692		ether_demux(ifp, m);
 (kgdb) p m
 $47 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $48 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ifp
 $49 =3D (struct ifnet *) 0xffffff00011a9800
 (kgdb) p *ifp
 $50 =3D {if_softc =3D 0xffffff8000211000, if_l2com =3D 0xffffff00012d97c0,=
 =20
   if_link =3D {tqe_next =3D 0xffffff00013c1000, tqe_prev =3D 0xffffff000116=
 b810},=20
   if_xname =3D "PORT1\000\000\000\000\000\000\000\000\000\000",=20
   if_dname =3D 0xffffff000117c9d8 "em", if_dunit =3D 1, if_addrhead =3D {
     tqh_first =3D 0xffffff00012e5800, tqh_last =3D 0xffffff00012e58b8},=20
   if_klist =3D {kl_list =3D {slh_first =3D 0x0},=20
     kl_lock =3D 0xffffffff803d8b40 <knlist_mtx_lock>,=20
     kl_unlock =3D 0xffffffff803d8590 <knlist_mtx_unlock>,=20
     kl_locked =3D 0xffffffff803d8570 <knlist_mtx_locked>,=20
     kl_lockarg =3D 0xffffffff809772e0}, if_pcount =3D 250, if_carp =3D 0x0,=
 =20
   if_bpf =3D 0xffffff00012ddd00, if_index =3D 2, if_timer =3D 0,=20
   if_vlantrunk =3D 0xffffff000c6cca00, if_flags =3D 35075, if_capabilities =
 =3D 475,=20
   if_capenable =3D 475, if_linkmib =3D 0x0, if_linkmiblen =3D 0, if_data =
 =3D {
     ifi_type =3D 6 '\006', ifi_physical =3D 0 '\0', ifi_addrlen =3D 6 '\006=
 ',=20
     ifi_hdrlen =3D 18 '\022', ifi_link_state =3D 2 '\002',=20
     ifi_spare_char1 =3D 0 '\0', ifi_spare_char2 =3D 0 '\0',=20
     ifi_datalen =3D 152 '\230', ifi_mtu =3D 1500, ifi_metric =3D 0,=20
     ifi_baudrate =3D 1000000000, ifi_ipackets =3D 2441940, ifi_ierrors =3D =
 677,=20
     ifi_opackets =3D 3951735, ifi_oerrors =3D 0, ifi_collisions =3D 0,=20
     ifi_ibytes =3D 385014266, ifi_obytes =3D 5239057455, ifi_imcasts =3D 38=
 496,=20
     ifi_omcasts =3D 16, ifi_iqdrops =3D 0, ifi_noproto =3D 4, ifi_hwassist =
 =3D 38,=20
     ifi_epoch =3D 1, ifi_lastchange =3D {tv_sec =3D 1251173561, tv_usec =3D=
  705068}},=20
   if_multiaddrs =3D {tqh_first =3D 0xffffff000c91bb40,=20
     tqh_last =3D 0xffffff000c91bb40}, if_amcount =3D 0,=20
   if_output =3D 0xffffffff804c7f40 <ether_output>,=20
   if_input =3D 0xffffffff804c8610 <ether_input>,=20
   if_start =3D 0xffffffff802570e0 <em_start>,=20
   if_ioctl =3D 0xffffffff80258de0 <em_ioctl>, if_watchdog =3D 0,=20
   if_init =3D 0xffffffff80255d40 <em_init>,=20
   if_resolvemulti =3D 0xffffffff804c72a0 <ether_resolvemulti>,=20
   if_addr =3D 0xffffff00012e5800, if_llsoftc =3D 0x0, if_drv_flags =3D 64,=
 =20
   if_spare_flags2 =3D 0, if_snd =3D {ifq_head =3D 0x0, ifq_tail =3D 0x0, if=
 q_len =3D 0,=20
     ifq_maxlen =3D 255, ifq_drops =3D 0, ifq_mtx =3D {lock_object =3D {
         lo_name =3D 0xffffff00011a9820 "PORT1",=20
         lo_type =3D 0xffffffff80743980 "if send queue", lo_flags =3D 169738=
 24,=20
         lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
       mtx_lock =3D 4, mtx_recurse =3D 0}, ifq_drv_head =3D 0x0, ifq_drv_tai=
 l =3D 0x0,=20
     ifq_drv_len =3D 0, ifq_drv_maxlen =3D 255, altq_type =3D 0, altq_flags =
 =3D 1,=20
     altq_disc =3D 0x0, altq_ifp =3D 0xffffff00011a9800, altq_enqueue =3D 0,=
 =20
     altq_dequeue =3D 0, altq_request =3D 0, altq_clfier =3D 0x0, altq_class=
 ify =3D 0,=20
     altq_tbr =3D 0x0, altq_cdnr =3D 0x0},=20
   if_broadcastaddr =3D 0xffffffff80744260 "=FF=FF=FF=FF=FF=FF", if_bridge =
 =3D 0x0,=20
   lltables =3D 0x0, if_label =3D 0x0, if_prefixhead =3D {tqh_first =3D 0x0,=
 =20
     tqh_last =3D 0xffffff00011a9a98}, if_afdata =3D {0x0 <repeats 28 times>=
 ,=20
     0xffffff000136d2e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},=20
   if_afdata_initialized =3D 2, if_afdata_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff80743976 "if_afdata",=20
       lo_type =3D 0xffffffff80743976 "if_afdata", lo_flags =3D 16973824,=20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_starttask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804ba8f0 <if_start_deferred>,=20
     ta_context =3D 0xffffff00011a9800}, if_linktask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804be2b0 <do_link_state_change>,=20
     ta_context =3D 0xffffff00011a9800}, if_addr_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff8074396a "if_addr_mtx",=20
       lo_type =3D 0xffffffff8074396a "if_addr_mtx", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_clones =3D {le_next =3D 0x0,=20
     le_prev =3D 0x0}, if_groups =3D {tqh_first =3D 0xffffff00012dbb80,=20
     tqh_last =3D 0xffffff00012dbb88}, if_pf_kif =3D 0x0, if_lagg =3D 0x0,=
 =20
   if_pspare =3D {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_ispa=
 re =3D {
     0, 0}}
 (kgdb) f 19
 #19 0xffffffff80257b02 in em_rxeof (adapter=3D0xffffff8000211000, count=3D2)
     at /usr/src2/src/sys/dev/e1000/if_em.c:4539
 4539				(*ifp->if_input)(ifp, m);
 (kgdb) p m
 $51 =3D (struct mbuf *) 0xffffff00013b6100
 (kgdb) p *m
 $52 =3D {m_hdr =3D {mh_next =3D 0x0, mh_nextpkt =3D 0x0,=20
     mh_data =3D 0xffffff000cc9d010 "E", mh_len =3D 40, mh_flags =3D 3, mh_t=
 ype =3D 1,=20
     pad =3D "\000\000\000\000\000"}, M_dat =3D {MH =3D {MH_pkthdr =3D {
         rcvif =3D 0xffffff000c771000, header =3D 0x0, len =3D 40, csum_flag=
 s =3D 768,=20
         csum_data =3D 0, tso_segsz =3D 0, ether_vtag =3D 91, tags =3D {
           slh_first =3D 0x0}}, MH_dat =3D {MH_ext =3D {
           ext_buf =3D 0xffffff000cc9d000 "=FF=FF\002", ext_free =3D 0, ext_=
 args =3D 0x0,=20
           ext_size =3D 2048, ref_cnt =3D 0xffffff000c9c79b4, ext_type =3D 6=
 },=20
         MH_databuf =3D "\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>=
 , "\b\000\000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0=
 \r[\217=E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\=
 217=E7=FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=
 =DBj\025\202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\20=
 3z=F8\021%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230#,=B2 \017=C2=
 =B7=E4=AA"}},=20
     M_databuf =3D "\000\020w\f\000=FF=FF=FF\000\000\000\000\000\000\000\000=
 (\000\000\000\000\003\000\000\000\000\000\000\000\000[\000\000\000\000\000\=
 000\000\000\000\000=D0=C9\f\000=FF=FF=FF", '\0' <repeats 17 times>, "\b\000=
 \000*\002K=BD=B4y\234\f\000=FF=FF=FF\006\000\000\000=BD=E9=D3eB=C0\r[\217=
 =E1r`?=EB\205=E4B=3D\020=B7=DA2=EC=E1!\234=B7vw\214=C9L\003\220\027\217=E7=
 =FAC6=A3=BA\226=FFx=D6&\r\217=A4=3D\005? \225=C1\206]=DF=E6cW\225\v=DBj\025=
 \202=D3=D2cr\210\\Y=D07\235\206=A2\016t(=BF=DF0\213\233\232V\006\203z=F8\02=
 1%hwt\037EowsR=D6=E8~t=DAN=DCd|\003Z=BC?=B5>=C5u\230"...}}
 (kgdb) p ifp
 $53 =3D (struct ifnet *) 0xffffff00011a9800
 (kgdb) p *ifp
 $54 =3D {if_softc =3D 0xffffff8000211000, if_l2com =3D 0xffffff00012d97c0,=
 =20
   if_link =3D {tqe_next =3D 0xffffff00013c1000, tqe_prev =3D 0xffffff000116=
 b810},=20
   if_xname =3D "PORT1\000\000\000\000\000\000\000\000\000\000",=20
   if_dname =3D 0xffffff000117c9d8 "em", if_dunit =3D 1, if_addrhead =3D {
     tqh_first =3D 0xffffff00012e5800, tqh_last =3D 0xffffff00012e58b8},=20
   if_klist =3D {kl_list =3D {slh_first =3D 0x0},=20
     kl_lock =3D 0xffffffff803d8b40 <knlist_mtx_lock>,=20
     kl_unlock =3D 0xffffffff803d8590 <knlist_mtx_unlock>,=20
     kl_locked =3D 0xffffffff803d8570 <knlist_mtx_locked>,=20
     kl_lockarg =3D 0xffffffff809772e0}, if_pcount =3D 250, if_carp =3D 0x0,=
 =20
   if_bpf =3D 0xffffff00012ddd00, if_index =3D 2, if_timer =3D 0,=20
   if_vlantrunk =3D 0xffffff000c6cca00, if_flags =3D 35075, if_capabilities =
 =3D 475,=20
   if_capenable =3D 475, if_linkmib =3D 0x0, if_linkmiblen =3D 0, if_data =
 =3D {
     ifi_type =3D 6 '\006', ifi_physical =3D 0 '\0', ifi_addrlen =3D 6 '\006=
 ',=20
     ifi_hdrlen =3D 18 '\022', ifi_link_state =3D 2 '\002',=20
     ifi_spare_char1 =3D 0 '\0', ifi_spare_char2 =3D 0 '\0',=20
     ifi_datalen =3D 152 '\230', ifi_mtu =3D 1500, ifi_metric =3D 0,=20
     ifi_baudrate =3D 1000000000, ifi_ipackets =3D 2441940, ifi_ierrors =3D =
 677,=20
     ifi_opackets =3D 3951735, ifi_oerrors =3D 0, ifi_collisions =3D 0,=20
     ifi_ibytes =3D 385014266, ifi_obytes =3D 5239057455, ifi_imcasts =3D 38=
 496,=20
     ifi_omcasts =3D 16, ifi_iqdrops =3D 0, ifi_noproto =3D 4, ifi_hwassist =
 =3D 38,=20
     ifi_epoch =3D 1, ifi_lastchange =3D {tv_sec =3D 1251173561, tv_usec =3D=
  705068}},=20
   if_multiaddrs =3D {tqh_first =3D 0xffffff000c91bb40,=20
     tqh_last =3D 0xffffff000c91bb40}, if_amcount =3D 0,=20
   if_output =3D 0xffffffff804c7f40 <ether_output>,=20
   if_input =3D 0xffffffff804c8610 <ether_input>,=20
   if_start =3D 0xffffffff802570e0 <em_start>,=20
   if_ioctl =3D 0xffffffff80258de0 <em_ioctl>, if_watchdog =3D 0,=20
   if_init =3D 0xffffffff80255d40 <em_init>,=20
   if_resolvemulti =3D 0xffffffff804c72a0 <ether_resolvemulti>,=20
   if_addr =3D 0xffffff00012e5800, if_llsoftc =3D 0x0, if_drv_flags =3D 64,=
 =20
   if_spare_flags2 =3D 0, if_snd =3D {ifq_head =3D 0x0, ifq_tail =3D 0x0, if=
 q_len =3D 0,=20
     ifq_maxlen =3D 255, ifq_drops =3D 0, ifq_mtx =3D {lock_object =3D {
         lo_name =3D 0xffffff00011a9820 "PORT1",=20
         lo_type =3D 0xffffffff80743980 "if send queue", lo_flags =3D 169738=
 24,=20
         lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
       mtx_lock =3D 4, mtx_recurse =3D 0}, ifq_drv_head =3D 0x0, ifq_drv_tai=
 l =3D 0x0,=20
     ifq_drv_len =3D 0, ifq_drv_maxlen =3D 255, altq_type =3D 0, altq_flags =
 =3D 1,=20
     altq_disc =3D 0x0, altq_ifp =3D 0xffffff00011a9800, altq_enqueue =3D 0,=
 =20
     altq_dequeue =3D 0, altq_request =3D 0, altq_clfier =3D 0x0, altq_class=
 ify =3D 0,=20
     altq_tbr =3D 0x0, altq_cdnr =3D 0x0},=20
   if_broadcastaddr =3D 0xffffffff80744260 "=FF=FF=FF=FF=FF=FF", if_bridge =
 =3D 0x0,=20
   lltables =3D 0x0, if_label =3D 0x0, if_prefixhead =3D {tqh_first =3D 0x0,=
 =20
     tqh_last =3D 0xffffff00011a9a98}, if_afdata =3D {0x0 <repeats 28 times>=
 ,=20
     0xffffff000136d2e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},=20
   if_afdata_initialized =3D 2, if_afdata_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff80743976 "if_afdata",=20
       lo_type =3D 0xffffffff80743976 "if_afdata", lo_flags =3D 16973824,=20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_starttask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804ba8f0 <if_start_deferred>,=20
     ta_context =3D 0xffffff00011a9800}, if_linktask =3D {ta_link =3D {
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff804be2b0 <do_link_state_change>,=20
     ta_context =3D 0xffffff00011a9800}, if_addr_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffffff8074396a "if_addr_mtx",=20
       lo_type =3D 0xffffffff8074396a "if_addr_mtx", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, if_clones =3D {le_next =3D 0x0,=20
     le_prev =3D 0x0}, if_groups =3D {tqh_first =3D 0xffffff00012dbb80,=20
     tqh_last =3D 0xffffff00012dbb88}, if_pf_kif =3D 0x0, if_lagg =3D 0x0,=
 =20
   if_pspare =3D {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, if_ispa=
 re =3D {
     0, 0}}
 (kgdb) p adapter
 $55 =3D (struct adapter *) 0xffffff8000211000
 (kgdb) p *adapter
 $56 =3D {ifp =3D 0xffffff00011a9800, hw =3D {back =3D 0xffffff8000215318,=
 =20
     hw_addr =3D 0xffffff8000215320 "", flash_address =3D 0x0, io_base =3D 0=
 , mac =3D {
       ops =3D {init_params =3D 0xffffffff802669c0 <e1000_init_mac_params_82=
 571>,=20
         blink_led =3D 0xffffffff80275c90 <e1000_blink_led_generic>,=20
         check_for_link =3D 0xffffffff80273bc0 <e1000_check_for_copper_link_=
 generic>, check_mng_mode =3D 0xffffffff80278250 <e1000_check_mng_mode_gener=
 ic>,=20
         cleanup_led =3D 0xffffffff80275d30 <e1000_cleanup_led_generic>,=20
         clear_hw_cntrs =3D 0xffffffff80267560 <e1000_clear_hw_cntrs_82571>,=
 =20
         clear_vfta =3D 0xffffffff80268b60 <e1000_clear_vfta_82571>,=20
         get_bus_info =3D 0xffffffff80273cf0 <e1000_get_bus_info_pcie_generi=
 c>,=20
         set_lan_id =3D 0xffffffff802730c0 <e1000_set_lan_id_single_port>,=
 =20
         get_link_up_info =3D 0xffffffff80274440 <e1000_get_speed_and_duplex=
 _copper_generic>, led_on =3D 0xffffffff80275b90 <e1000_led_on_generic>,=20
         led_off =3D 0xffffffff80275aa0 <e1000_led_off_generic>,=20
         update_mc_addr_list =3D 0xffffffff80267f00 <e1000_update_mc_addr_li=
 st_82571>, reset_hw =3D 0xffffffff80269570 <e1000_reset_hw_82571>,=20
         init_hw =3D 0xffffffff80268e00 <e1000_init_hw_82571>,=20
         shutdown_serdes =3D 0,=20
         setup_link =3D 0xffffffff80267f70 <e1000_setup_link_82571>,=20
         setup_physical_interface =3D 0xffffffff802688d0 <e1000_setup_copper=
 _link_82571>, setup_led =3D 0xffffffff80275db0 <e1000_setup_led_generic>,=
 =20
         write_vfta =3D 0xffffffff802751e0 <e1000_write_vfta_generic>,=20
         mta_set =3D 0xffffffff802774c0 <e1000_mta_set_generic>,=20
         config_collision_dist =3D 0xffffffff802738c0 <e1000_config_collisio=
 n_dist_generic>, rar_set =3D 0xffffffff80277610 <e1000_rar_set_generic>,=20
         read_mac_addr =3D 0xffffffff80267ee0 <e1000_read_mac_addr_82571>,=
 =20
         validate_mdi_setting =3D 0xffffffff802733a0 <e1000_validate_mdi_set=
 ting_generic>,=20
         mng_host_if_write =3D 0xffffffff802783d0 <e1000_mng_host_if_write_g=
 eneric>,=20
         mng_write_cmd_header =3D 0xffffffff80277f00 <e1000_mng_write_cmd_he=
 ader_generic>, mng_enable_host_if =3D 0xffffffff802779e0 <e1000_mng_enable_=
 host_if_generic>,=20
         wait_autoneg =3D 0xffffffff8027c1b0 <e1000_wait_autoneg_generic>},=
 =20
       addr =3D "\0000H=B11=C7", perm_addr =3D "\0000H=B11=C7", type =3D e10=
 00_82573,=20
       collision_delta =3D 0, ledctl_default =3D 492550, ledctl_mode1 =3D 10=
 16838,=20
       ledctl_mode2 =3D 951302, mc_filter_type =3D 0, tx_packet_delta =3D 0,=
 =20
       txcw =3D 0, current_ifs_val =3D 0, ifs_max_val =3D 0, ifs_min_val =3D=
  0,=20
       ifs_ratio =3D 0, ifs_step_size =3D 0, mta_reg_count =3D 128,=20
       rar_entry_count =3D 15, forced_speed_duplex =3D 0 '\0', adaptive_ifs =
 =3D 0,=20
       arc_subsystem_valid =3D 0, asf_firmware_present =3D 1, autoneg =3D 1,=
 =20
       autoneg_failed =3D 0, get_link_status =3D 0, in_ifs_mode =3D 0,=20
       report_tx_early =3D 1, serdes_has_link =3D 0, tx_pkt_filtering =3D 0}=
 , fc =3D {
       high_water =3D 10240, low_water =3D 8740, pause_time =3D 1664, send_x=
 on =3D 1,=20
       strict_ieee =3D 0, current_mode =3D e1000_fc_none,=20
       requested_mode =3D e1000_fc_full}, phy =3D {ops =3D {
         init_params =3D 0xffffffff80266710 <e1000_init_phy_params_82571>,=
 =20
         acquire =3D 0xffffffff80266c60 <e1000_get_hw_semaphore_82571>,=20
         cfg_on_link_up =3D 0xffffffff80273040 <e1000_null_ops_generic>,=20
         check_polarity =3D 0xffffffff8027b6a0 <e1000_check_polarity_igp>,=
 =20
         check_reset_block =3D 0xffffffff8027d790 <e1000_check_reset_block_g=
 eneric>, commit =3D 0xffffffff8027be00 <e1000_phy_sw_reset_generic>,=20
         force_speed_duplex =3D 0xffffffff8027d480 <e1000_phy_force_speed_du=
 plex_m88>, get_cfg_done =3D 0xffffffff8027bde0 <e1000_get_cfg_done_generic>=
 ,=20
         get_cable_length =3D 0xffffffff8027b740 <e1000_get_cable_length_m88=
 >,=20
         get_info =3D 0xffffffff8027c070 <e1000_get_phy_info_m88>,=20
         read_reg =3D 0xffffffff8027d2e0 <e1000_read_phy_reg_m88>,=20
         release =3D 0xffffffff80266910 <e1000_put_hw_semaphore_82571>,=20
         reset =3D 0xffffffff8027d7e0 <e1000_phy_hw_reset_generic>,=20
         set_d0_lplu_state =3D 0xffffffff80266420 <e1000_set_d0_lplu_state_8=
 2571>, set_d3_lplu_state =3D 0xffffffff8027b4a0 <e1000_set_d3_lplu_state_ge=
 neric>,=20
         write_reg =3D 0xffffffff8027cae0 <e1000_write_phy_reg_m88>,=20
         power_up =3D 0xffffffff8027bd50 <e1000_power_up_phy_copper>,=20
         power_down =3D 0xffffffff80266c20 <e1000_power_down_phy_copper_8257=
 1>},=20
       type =3D e1000_phy_m88, local_rx =3D e1000_1000t_rx_status_not_ok,=20
       remote_rx =3D e1000_1000t_rx_status_not_ok, ms_type =3D e1000_ms_hw_d=
 efault,=20
       original_ms_type =3D e1000_ms_hw_default,=20
 ---Type <return> to continue, or q <return> to quit---
       cable_polarity =3D e1000_rev_polarity_normal,=20
       smart_speed =3D e1000_smart_speed_default, addr =3D 1, id =3D 2104032=
 0,=20
       reset_delay_us =3D 100, revision =3D 2,=20
       media_type =3D e1000_media_type_copper, autoneg_advertised =3D 47,=20
       autoneg_mask =3D 47, cable_length =3D 0, max_cable_length =3D 0,=20
       min_cable_length =3D 0, mdix =3D 0 '\0', disable_polarity_correction =
 =3D 0,=20
       is_mdix =3D 0, polarity_correction =3D 0, reset_disable =3D 1,=20
       speed_downgraded =3D 0, autoneg_wait_to_complete =3D 0}, nvm =3D {ops=
  =3D {
         init_params =3D 0xffffffff802681b0 <e1000_init_nvm_params_82571>,=
 =20
         acquire =3D 0xffffffff80267500 <e1000_acquire_nvm_82571>,=20
         read =3D 0xffffffff8027aba0 <e1000_read_nvm_eerd>,=20
         release =3D 0xffffffff802674e0 <e1000_release_nvm_82571>,=20
         reload =3D 0xffffffff8027aaa0 <e1000_reload_nvm_generic>,=20
         update =3D 0xffffffff802683a0 <e1000_update_nvm_checksum_82571>,=20
         valid_led_default =3D 0xffffffff80266570 <e1000_valid_led_default_8=
 2571>, validate =3D 0xffffffff80267430 <e1000_validate_nvm_checksum_82571>,=
 =20
         write =3D 0xffffffff80267100 <e1000_write_nvm_82571>},=20
       type =3D e1000_nvm_eeprom_spi, override =3D e1000_nvm_override_none,=
 =20
       flash_bank_size =3D 0, flash_base_addr =3D 0, word_size =3D 64,=20
       delay_usec =3D 1, address_bits =3D 8, opcode_bits =3D 8, page_size =
 =3D 8},=20
     bus =3D {type =3D e1000_bus_type_pci_express, speed =3D e1000_bus_speed=
 _2500,=20
       width =3D 56, func =3D 0, pci_cmd_word =3D 263}, mng_cookie =3D {sign=
 ature =3D 0,=20
       status =3D 0 '\0', reserved0 =3D 0 '\0', vlan_id =3D 0, reserved1 =3D=
  0,=20
       reserved2 =3D 0, reserved3 =3D 0 '\0', checksum =3D 0 '\0'}, dev_spec=
  =3D {
       _82541 =3D {dsp_config =3D e1000_dsp_config_disabled,=20
         ffe_config =3D e1000_ffe_config_enabled, spd_default =3D 0,=20
         phy_init_script =3D 0}, _82542 =3D {dma_fairness =3D 0}, _82543 =3D=
  {
         tbi_compatibility =3D 0, dma_fairness =3D 0, init_phy_disabled =3D =
 0},=20
       _82571 =3D {laa_is_present =3D 0}, ich8lan =3D {
         kmrn_lock_loss_workaround_enabled =3D 0, shadow_ram =3D {{value =3D=
  0,=20
             modified =3D 0} <repeats 2048 times>}}, _82575 =3D {sgmii_activ=
 e =3D 0},=20
       vf =3D {vf_number =3D 0}}, device_id =3D 4250, subsystem_vendor_id =
 =3D 5593,=20
     subsystem_device_id =3D 4250, vendor_id =3D 32902, revision_id =3D 0 '\=
 0'},=20
   osdep =3D {mem_bus_space_tag =3D 1, mem_bus_space_handle =3D 184467435239=
 55433472,=20
     io_bus_space_tag =3D 0, io_bus_space_handle =3D 0, flash_bus_space_tag =
 =3D 0,=20
     flash_bus_space_handle =3D 0, dev =3D 0xffffff0001292800},=20
   dev =3D 0xffffff0001292800, memory =3D 0xffffff00012dd500, flash =3D 0x0,=
 =20
   msix =3D 0x0, ioport =3D 0x0, io_rid =3D 0, res =3D {0xffffff00012dcb80, =
 0x0, 0x0},=20
   tag =3D {0xffffff00012dd900, 0x0, 0x0}, rid =3D {1, 2, 3}, media =3D {
     ifm_mask =3D -268435456, ifm_media =3D 0, ifm_cur =3D 0xffffff00012dba6=
 0,=20
     ifm_list =3D {lh_first =3D 0xffffff00012dba60},=20
     ifm_change =3D 0xffffffff80258750 <em_media_change>,=20
     ifm_status =3D 0xffffffff802588e0 <em_media_status>}, timer =3D {c_link=
 s =3D {
       sle =3D {sle_next =3D 0xffffff000118e540}, tqe =3D {
         tqe_next =3D 0xffffff000118e540, tqe_prev =3D 0xffffff80148267f0}},=
 =20
     c_time =3D 1210485, c_arg =3D 0xffffff8000211000,=20
     c_func =3D 0xffffffff80255de0 <em_local_timer>, c_mtx =3D 0xffffff80002=
 15470,=20
     c_flags =3D 6}, tx_fifo_timer =3D {c_links =3D {sle =3D {sle_next =3D 0=
 x0}, tqe =3D {
         tqe_next =3D 0x0, tqe_prev =3D 0x0}}, c_time =3D 0, c_arg =3D 0x0, =
 c_func =3D 0,=20
     c_mtx =3D 0xffffff80002154a0, c_flags =3D 0}, watchdog_timer =3D 5, msi=
  =3D 1,=20
   if_flags =3D 35075, max_frame_size =3D 1518, min_frame_size =3D 64, core_=
 mtx =3D {
     lock_object =3D {lo_name =3D 0xffffff00012cd960 "em1",=20
       lo_type =3D 0xffffffff8071484a "EM Core Lock", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, tx_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffff00012cd960 "em1",=20
       lo_type =3D 0xffffffff80714857 "EM TX Lock", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, rx_mtx =3D {lock_object =3D {
       lo_name =3D 0xffffff00012cd960 "em1",=20
       lo_type =3D 0xffffffff80714862 "EM RX Lock", lo_flags =3D 16973824,=
 =20
       lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness =
 =3D 0x0}},=20
     mtx_lock =3D 4, mtx_recurse =3D 0}, em_insert_vlan_header =3D 0, link_t=
 ask =3D {
     ta_link =3D {stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=
 =20
     ta_func =3D 0xffffffff80258c50 <em_handle_link>,=20
     ta_context =3D 0xffffff8000211000}, rxtx_task =3D {ta_link =3D {
 ---Type <return> to continue, or q <return> to quit---
       stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0,=20
     ta_func =3D 0xffffffff80258b60 <em_handle_rxtx>,=20
     ta_context =3D 0xffffff8000211000}, rx_task =3D {ta_link =3D {stqe_next=
  =3D 0x0},=20
     ta_pending =3D 0, ta_priority =3D 0, ta_func =3D 0, ta_context =3D 0x0}=
 ,=20
   tx_task =3D {ta_link =3D {stqe_next =3D 0x0}, ta_pending =3D 0, ta_priori=
 ty =3D 0,=20
     ta_func =3D 0, ta_context =3D 0x0}, tq =3D 0xffffff00012dd880, wol =3D =
 2,=20
   has_manage =3D 0, link_active =3D 1 '\001', link_speed =3D 1000, link_dup=
 lex =3D 2,=20
   smartspeed =3D 0, tx_int_delay =3D {adapter =3D 0xffffff8000211000,=20
     offset =3D 14368, value =3D 66}, tx_abs_int_delay =3D {
     adapter =3D 0xffffff8000211000, offset =3D 14380, value =3D 66}, rx_int=
 _delay =3D {
     adapter =3D 0xffffff8000211000, offset =3D 10272, value =3D 0},=20
   rx_abs_int_delay =3D {adapter =3D 0xffffff8000211000, offset =3D 10284,=
 =20
     value =3D 66}, txdma =3D {dma_paddr =3D 19423232,=20
     dma_vaddr =3D 0xffffff0001286000 "", dma_tag =3D 0xffffff00012dc880,=20
     dma_map =3D 0x0, dma_seg =3D {ds_addr =3D 0, ds_len =3D 0}, dma_nseg =
 =3D 0},=20
   tx_desc_base =3D 0xffffff0001286000, next_avail_tx_desc =3D 143,=20
   next_tx_to_clean =3D 138, num_tx_desc_avail =3D 251, num_tx_desc =3D 256,=
 =20
   txd_cmd =3D 2181038080, tx_buffer_area =3D 0xffffff8000216000,=20
   txtag =3D 0xffffff00012dd280, tx_tso =3D 0, rxdma =3D {dma_paddr =3D 1941=
 9136,=20
     dma_vaddr =3D 0xffffff0001285000 "\002\200\225\f",=20
     dma_tag =3D 0xffffff00012dd200, dma_map =3D 0x0, dma_seg =3D {ds_addr =
 =3D 0,=20
       ds_len =3D 0}, dma_nseg =3D 0}, rx_desc_base =3D 0xffffff0001285000,=
 =20
   next_rx_desc_to_check =3D 212, rx_buffer_len =3D 2048, num_rx_desc =3D 25=
 6,=20
   rx_process_limit =3D 100, rx_buffer_area =3D 0xffffff8000218000,=20
   rxtag =3D 0xffffff00012dd300, rx_sparemap =3D 0x0, fmp =3D 0x0, lmp =3D 0=
 x0,=20
   dropped_pkts =3D 0, mbuf_alloc_failed =3D 0, mbuf_cluster_failed =3D 0,=
 =20
   no_tx_desc_avail1 =3D 0, no_tx_desc_avail2 =3D 0, no_tx_map_avail =3D 0,=
 =20
   no_tx_dma_setup =3D 0, watchdog_events =3D 0, rx_overruns =3D 0, rx_irq =
 =3D 0,=20
   tx_irq =3D 0, link_irq =3D 0, tx_fifo_size =3D 0, tx_fifo_head =3D 0,=20
   tx_fifo_head_addr =3D 0, tx_fifo_reset_cnt =3D 0, tx_fifo_wrk_cnt =3D 0,=
 =20
   tx_head_addr =3D 0, pcix_82544 =3D 0, in_detach =3D 0, stats =3D {crcerrs=
  =3D 0,=20
     algnerrc =3D 0, symerrs =3D 0, rxerrc =3D 0, mpc =3D 677, scc =3D 0, ec=
 ol =3D 0,=20
     mcc =3D 0, latecol =3D 0, colc =3D 0, dc =3D 0, tncrs =3D 0, sec =3D 0,=
  cexterr =3D 0,=20
     rlec =3D 0, xonrxc =3D 0, xontxc =3D 0, xoffrxc =3D 0, xofftxc =3D 0, f=
 cruc =3D 0,=20
     prc64 =3D 657, prc127 =3D 2198382, prc255 =3D 29080, prc511 =3D 39931,=
 =20
     prc1023 =3D 37558, prc1522 =3D 135809, gprc =3D 2441417, bprc =3D 17185=
 ,=20
     mprc =3D 21302, gptc =3D 3951004, gorc =3D 0, gotc =3D 0, rnbc =3D 337,=
  ruc =3D 0,=20
     rfc =3D 0, roc =3D 0, rjc =3D 0, mgprc =3D 0, mgpdc =3D 0, mgptc =3D 0,=
  tor =3D 0,=20
     tot =3D 0, tpr =3D 2442094, tpt =3D 3951004, ptc64 =3D 130645, ptc127 =
 =3D 150183,=20
     ptc255 =3D 31106, ptc511 =3D 73338, ptc1023 =3D 33581, ptc1522 =3D 3532=
 151,=20
     mptc =3D 16, bptc =3D 96077, tsctc =3D 0, tsctfc =3D 0, iac =3D 0, icrx=
 ptc =3D 0,=20
     icrxatc =3D 0, ictxptc =3D 0, ictxatc =3D 0, ictxqec =3D 0, ictxqmtc =
 =3D 0,=20
     icrxdmtc =3D 0, icrxoc =3D 0, cbtmpc =3D 0, htdpmc =3D 0, cbrdpc =3D 0,=
  cbrmpc =3D 0,=20
     rpthc =3D 0, hgptc =3D 0, htcbdpc =3D 0, hgorc =3D 0, hgotc =3D 0, lene=
 rrs =3D 0,=20
     scvpc =3D 0, hrmpc =3D 0, doosync =3D 0}}
 (kgdb) p count
 $57 =3D 2
 (kgdb) BUILD-7# ^D=08=08exit
 
 Script done on Mon Aug 31 16:21:11 2009
 
 --LZvS9be/3tNcYl/X--
State-Changed-From-To: feedback->open 
State-Changed-By: gavin 
State-Changed-When: Tue Sep 22 13:43:56 UTC 2009 
State-Changed-Why:  
Lots of good feedback provided by submitter 


Responsible-Changed-From-To: gavin->freebsd-net 
Responsible-Changed-By: gavin 
Responsible-Changed-When: Tue Sep 22 13:43:56 UTC 2009 
Responsible-Changed-Why:  
Pass this over to -net.  I've dropped darrenr@ emails without response. 
Hopefully with all of the debugging information provided, somebody with 
more knowledge about the networking stack will be able to resolve this. 

There's also a possibility that the patch in PR 139058 might help, I've 
suggested that the submitter test that if he is able to. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=138177 
Responsible-Changed-From-To: freebsd-net->cy 
Responsible-Changed-By: cy 
Responsible-Changed-When: Wed Jul 3 05:21:35 UTC 2013 
Responsible-Changed-Why:  
Mine. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=138177 
>Unformatted:
