From mainland@apeiron.net  Wed Sep 17 12:44:45 2008
Return-Path: <mainland@apeiron.net>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 66200106564A
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 17 Sep 2008 12:44:45 +0000 (UTC)
	(envelope-from mainland@apeiron.net)
Received: from zeno.apeiron.net (h-68-164-219-98.cmbrmaor.covad.net [68.164.219.98])
	by mx1.freebsd.org (Postfix) with ESMTP id 490048FC13
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 17 Sep 2008 12:44:44 +0000 (UTC)
	(envelope-from mainland@apeiron.net)
Received: from zeno.apeiron.net (localhost [127.0.0.1])
	by zeno.apeiron.net (Postfix) with ESMTP id 1CED422CCC
	for <FreeBSD-gnats-submit@freebsd.org>; Wed, 17 Sep 2008 08:33:24 -0400 (EDT)
Received: by zeno.apeiron.net (Postfix, from userid 4001)
	id EFF8F22CCB; Wed, 17 Sep 2008 08:33:23 -0400 (EDT)
Message-Id: <20080917123323.EFF8F22CCB@zeno.apeiron.net>
Date: Wed, 17 Sep 2008 08:33:23 -0400 (EDT)
From: Geoffrey Mainland <mainland@apeiron.net>
Reply-To: Geoffrey Mainland <mainland@apeiron.net>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: deadlock in pf
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         127439
>Category:       kern
>Synopsis:       [pf] deadlock in pf
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    glebius
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Sep 17 12:50:01 UTC 2008
>Closed-Date:    Wed Jan 22 09:17:10 UTC 2014
>Last-Modified:  Wed Jan 22 09:17:10 UTC 2014
>Originator:     Geoffrey Mainland
>Release:        FreeBSD 7.1-PRERELEASE i386
>Organization:
>Environment:
System: FreeBSD zeno.apeiron.net 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #7: Tue Sep 16 09:28:16 EDT 2008 toor@zeno.apeiron.net:/usr/obj/usr/src/sys/ZENO i386


>Description:

This happens reliably every night. I'm not sure what's running that triggers it.

ifconfig:

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:0e:0c:5f:c1:f8
        inet6 fe80::20e:cff:fe5f:c1f8%em0 prefixlen 64 scopeid 0x1 
        inet 192.168.0.10 netmask 0xffffff00 broadcast 192.168.0.255
        inet 192.168.0.1 netmask 0xffffffff broadcast 192.168.0.1
        inet 192.168.0.2 netmask 0xffffffff broadcast 192.168.0.2
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:90:27:62:87:4d
        inet6 fe80::290:27ff:fe62:874d%fxp0 prefixlen 64 scopeid 0x2 
        inet 68.164.219.98 netmask 0xfffffff8 broadcast 68.164.219.103
        inet 68.164.219.99 netmask 0xffffffff broadcast 68.164.219.99
        inet 68.164.219.100 netmask 0xffffffff broadcast 68.164.219.100
        inet 68.164.219.101 netmask 0xffffffff broadcast 68.164.219.101
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2808<VLAN_MTU,WOL_UCAST,WOL_MAGIC>
        ether 00:15:f2:43:48:7b
        inet6 fe80::215:f2ff:fe43:487b%vr0 prefixlen 64 scopeid 0x3 
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet 192.168.1.2 netmask 0xffffffff broadcast 192.168.1.2
        media: Ethernet autoselect (none)
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
        inet 127.0.0.1 netmask 0xff000000 
pfsync0: flags=0<> metric 0 mtu 1460
        syncpeer: 224.0.0.240 maxupd: 128
pflog0: flags=0<> metric 0 mtu 33204
gif0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1280
        tunnel inet 68.164.219.98 --> 66.55.128.25
        inet6 fe80::20e:cff:fe5f:c1f8%gif0 prefixlen 64 scopeid 0x7 
        inet6 2001:4830:1200:10b::2 --> 2001:4830:1200:10b::1 prefixlen 128 
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        inet6 fe80::20e:cff:fe5f:c1f8%tun0 prefixlen 64 scopeid 0x8 
        inet 192.168.2.1 --> 192.168.2.2 netmask 0xffffffff 
        Opened by PID 1454

Kernel config:

cpu		I686_CPU
ident		ZENO
options 	SCHED_ULE
options		SMP
options         PREEMPTION
options         DEVICE_POLLING
options		HZ=2000
options		_KPOSIX_PRIORITY_SCHEDULING
options		P1003_1B_MQUEUE
options		KDB
options		KDB_TRACE
options		DDB
options		WITNESS
options		INVARIANTS
options		INVARIANT_SUPPORT
makeoptions	DEBUG=-g		#Build kernel with gdb(1) debug symbols
options 	COMPAT_FREEBSD4
options 	COMPAT_FREEBSD5
options 	COMPAT_FREEBSD6
options 	SYSVSHM
options 	SYSVSEM
options 	SYSVMSG
options 	STACK
options 	INET			#Internet communications protocols
options 	INET6			#IPv6 communications protocols
options 	IPSEC			#IP security (requires device crypto)
options 	NETATALK		#Appletalk communications protocols
options 	NETSMB			#SMB/CIFS requester
options 	LIBMCHAIN
options         SCTP
options 	NETGRAPH		# netgraph(4) system
device		ether			#Generic Ethernet
device		loop			#Network loopback device
device		bpf			#Berkeley packet filter
device		tap			#Virtual Ethernet driver
device		tun			#Tunnel driver (ppp(8), nos-tun(8))
device		gre			#IP over IP tunneling
device		pf			#PF OpenBSD packet-filter firewall
device		pflog			#logging support interface for PF
device		pfsync			#synchronization interface for PF
device		gif			#IPv6 and IPv4 tunneling
device		faith			#for IPv6 and IPv4 translation
device		stf			#6to4 IPv6 over IPv4 encapsulation
options 	FFS			#Fast filesystem
options 	NFSCLIENT		#Network File System client
options 	CD9660			#ISO 9660 filesystem
options 	MSDOSFS			#MS DOS File System (FAT, FAT32)
options 	NFSSERVER		#Network File System server
options		NFSLOCKD		#Network Lock Manager
options 	NTFS			#NT File System
options 	PROCFS			#Process filesystem (requires PSEUDOFS)
options 	PSEUDOFS		#Pseudo-filesystem framework
options 	SMBFS			#SMB/CIFS filesystem
options 	UDF			#Universal Disk Format
options 	NFS_ROOT		#NFS usable as root device
options 	SOFTUPDATES
options 	UFS_ACL
options 	UFS_DIRHASH
device		random
device		mem
options 	AUDIT
device		scbus		#base SCSI code
device		da		#SCSI direct access devices (aka disks)
device		cd		#SCSI CD-ROMs
device		pt		#SCSI processor
device		pass		#CAM passthrough driver
device		pty		#Pseudo ttys
device		md		#Memory/malloc disk
options 	LIBICONV
options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
device		splash			# Splash screen and screen saver support
device		sc
options 	SC_DISABLE_KDBKEY	# disable `debug' key
device		ata
device		atadisk		# ATA disk drives
device		ataraid		# ATA RAID drives
device		atapicd		# ATAPI CDROM drives
device		atapifd		# ATAPI floppy drives
device		atapicam	# emulate ATAPI devices as SCSI ditto via CAM
options 	ATA_STATIC_ID
device		fdc
device		sound
device		ppc
device		ppbus
device		lpt
device		ppi
device		uhci
device		ehci
device		usb
device		crypto		# core crypto support
device		cryptodev	# /dev/crypto for access to h/w
device		apic			# I/O apic
device		nvram		# Access to rtc cmos via /dev/nvram
device		sio
device		eisa
device		pci
options 	VESA
device		psm
device		atkbdc
device		atkbd
device		vga
options 	COMPAT_LINUX
options 	COMPAT_AOUT
options 	LINPROCFS
options		LINSYSFS





dmesg output (after crash):

Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	  The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.1-PRERELEASE #7: Tue Sep 16 09:28:16 EDT 2008
    toor@zeno.apeiron.net:/usr/obj/usr/src/sys/ZENO
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Sempron(tm) Processor 3100+ (1800.09-MHz 686-class CPU)
  Origin = "AuthenticAMD"  Id = 0x10fc0  Stepping = 0
  Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
  AMD Features=0xc2500800<SYSCALL,NX,MMX+,FFXSR,3DNow!+,3DNow!>
  AMD Features2=0x1<LAHF>
real memory  = 1073414144 (1023 MB)
avail memory = 1040887808 (992 MB)
WITNESS: spin lock cpuset not in order list
WITNESS: spin lock intrcnt not in order list
netsmb_dev: loaded
cryptosoft0: <software crypto> on motherboard
acpi0: <A M I OEMRSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 3fef0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> mem
0xfb000000-0xfbffffff,0xf0000000-0xf7ffffff irq 11 at device 0.0 on pci1
em0: <Intel(R) PRO/1000 Network Connection 6.9.5> port 0xe800-0xe83f mem
0xfae00000-0xfae1ffff,0xfad00000-0xfad1ffff irq 11 at device 11.0 on pci0
em0: [FILTER]
em0: Ethernet address: 00:0e:0c:5f:c1:f8
fxp0: <Intel 82559 Pro/100 Ethernet> port 0xe400-0xe43f mem
0xfab00000-0xfab00fff,0xfaa00000-0xfaafffff irq 10 at device 12.0 on pci0
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> PHY 1 on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:90:27:62:87:4d
fxp0: [ITHREAD]
atapci0: <VIA 6420 SATA150 controller> port
0xe000-0xe007,0xd800-0xd803,0xd400-0xd407,0xd000-0xd003,0xc800-0xc80f,0xc400-0xc4ff
irq 10 at device 15.0 on pci0
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
atapci1: <VIA 8237 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 15.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci1
ata1: [ITHREAD]
uhci0: <VIA 83C572 USB controller> port 0xb000-0xb01f irq 11 at device 16.0 on
pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <VIA 83C572 USB controller> on uhci0
usb0: USB revision 1.0
uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <VIA 83C572 USB controller> port 0xb400-0xb41f irq 11 at device 16.1 on
pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <VIA 83C572 USB controller> on uhci1
usb1: USB revision 1.0
uhub1: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: <VIA 83C572 USB controller> port 0xb800-0xb81f irq 10 at device 16.2 on
pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: <VIA 83C572 USB controller> on uhci2
usb2: USB revision 1.0
uhub2: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
uhci3: <VIA 83C572 USB controller> port 0xc000-0xc01f irq 10 at device 16.3 on
pci0
uhci3: [GIANT-LOCKED]
uhci3: [ITHREAD]
usb3: <VIA 83C572 USB controller> on uhci3
usb3: USB revision 1.0
uhub3: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ehci0: <VIA VT6202 USB 2.0 controller> mem 0xfa700000-0xfa7000ff irq 5 at device
16.4 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <VIA VT6202 USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: <VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
uhub4: 8 ports with 8 removable, self powered
isab0: <PCI-ISA bridge> at device 17.0 on pci0
isa0: <ISA bus> on isab0
pci0: <multimedia, audio> at device 17.5 (no driver attached)
vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0xa400-0xa4ff mem
0xfa600000-0xfa6000ff irq 11 at device 18.0 on pci0
vr0: Quirks: 0x0
vr0: Revision: 0x78
miibus1: <MII bus> on vr0
rlphy0: <RTL8201L 10/100 media interface> PHY 1 on miibus1
rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr0: Ethernet address: 00:15:f2:43:48:7b
vr0: [ITHREAD]
cpu0: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
acpi_button1: <Sleep Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio0: [FILTER]
orm0: <ISA Option ROMs> at iomem 0xcd000-0xcdfff,0xce000-0xcefff,0xcf000-0xd3fff
pnpid ORM0000 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
ppbus0: <Parallel port bus> on ppc0
ppbus0: [ITHREAD]
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
Timecounter "TSC" frequency 1800086355 Hz quality 800
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
ad0: 194481MB <Maxtor 6B200P0 BAH41BM0> at ata0-master UDMA133
acd0: DVDR <NEC DVD RW ND-3550A/1.05> at ata1-master UDMA33
ad4: 239372MB <Maxtor 7L250S0 BANC1G10> at ata2-master SATA150
cd0 at ata1 bus 0 target 0 lun 0
cd0: <_NEC DVD_RW ND-3550A 1.05> Removable CD-ROM SCSI-0 device 
cd0: 33.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/ad4s1a
WARNING: / was not properly dismounted
lock order reversal:
 1st 0xc0907fcc pf task mtx (pf task mtx) @
 /usr/src/sys/contrib/pf/net/pf_ioctl.c:1394
 2nd 0xc0973488 ifnet (ifnet) @ /usr/src/sys/net/if.c:1558
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e658ba3c,c05eb7b6,c088f4ad,c0973488,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c088f4ad,c0973488,c0896cfd,c0896cfd,c0896b56,...) at
kdb_backtrace+0x29
witness_checkorder(c0973488,9,c0896b56,616,0,...) at witness_checkorder+0x6d6
_mtx_lock_flags(c0973488,0,c0896b56,616,c3f37a70,...) at _mtx_lock_flags+0xbc
ifunit(c3f37a70,0,c08711f2,572,c05e958e,...) at ifunit+0x2f
pfioctl(c3d2d800,c0104414,c3f37a70,3,c3f48690,...) at pfioctl+0x23b5
devfs_ioctl_f(c3f49c2c,c0104414,c3f37a70,c3b2c000,c3f48690,...) at
devfs_ioctl_f+0xe5
kern_ioctl(c3f48690,3,c0104414,c3f37a70,1000000,...) at kern_ioctl+0x243
ioctl(c3f48690,e658bcfc,c,c08bade8,c08d3630,...) at ioctl+0x134
syscall(e658bd38) at syscall+0x274
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (54, FreeBSD ELF32, ioctl), eip = 0x281aac4b, esp = 0xbfbfde5c, ebp
= 0xbfbfde88 ---
lock order reversal:
 1st 0xc097830c tcp (tcp) @ /usr/src/sys/netinet/tcp_input.c:400
 2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 /usr/src/sys/net/pfil.c:73
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e42579ac,c05eb7b6,c088f4ad,c09775d8,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
kdb_backtrace+0x29
witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
witness_checkorder+0x6d6
_rw_rlock(c09775d8,c0897d93,49,e4257a6c,0,...) at _rw_rlock+0x8e
pfil_run_hooks(c09775c0,e4257a8c,c3c31c00,2,0,...) at pfil_run_hooks+0x35
ip_output(c3c46100,0,e4257a50,0,0,0,c08e7c90,0,0,0,c067c807,c08e7c94,c08e7c9c,c8)
at ip_output+0x90f
tcp_respond(0,c3c87020,c3c87034,c3c46100,2da9088c,...) at tcp_respond+0x3e7
tcp_dropwithreset(1,3,c089c953,353,1900,...) at tcp_dropwithreset+0x152
tcp_input(c3c46100,14,c3c31c00,1,0,...) at tcp_input+0xe45
ip_input(c3c46100,c3c46100,800,c3c31c00,800,...) at ip_input+0x686
netisr_dispatch(2,c3c46100,10,3,0,...) at netisr_dispatch+0x72
ether_demux(c3c31c00,c3c46100,3,0,3,...) at ether_demux+0x2e5
ether_input(c3c31c00,c3c46100,c0aa0a74,6a9,ffffffff,...) at ether_input+0x37f
fxp_intr_body(ffffffff,0,c0aa0a74,5db,c3c33014,...) at fxp_intr_body+0x1c4
fxp_intr(c3c33000,0,c08866ae,4b6,c3b3c268,...) at fxp_intr+0xa0
ithread_loop(c3c1fa50,e4257d38,c0886453,31c,c3bef2b8,...) at ithread_loop+0x1c5
fork_exit(c0590660,c3c1fa50,e4257d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe4257d70, ebp = 0 ---
lock order reversal:
 1st 0xc4013d44 udpinp (udpinp) @ /usr/src/sys/netinet/udp_usrreq.c:878
 2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 /usr/src/sys/net/pfil.c:73
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e658ba14,c05eb7b6,c088f4ad,c09775d8,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
kdb_backtrace+0x29
witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
witness_checkorder+0x6d6
_rw_rlock(c09775d8,c0897d93,49,e658bad4,c4013ca8,...) at _rw_rlock+0x8e
pfil_run_hooks(c09775c0,e658baf4,c3d44000,2,c4013ca8,...) at pfil_run_hooks+0x35
ip_output(c3ef6100,0,e658bab8,0,0,...) at ip_output+0x90f
udp_send(c42454e0,0,c3ef6100,0,0,...) at udp_send+0x8cd
sosend_dgram(c42454e0,0,e658bbec,c3ef6100,0,...) at sosend_dgram+0x351
sosend(c42454e0,0,e658bbec,0,0,...) at sosend+0x54
kern_sendit(c3f48690,4,e658bc68,0,0,...) at kern_sendit+0xdb
sendit(0,8143023,0,0,0,...) at sendit+0xb1
sendto(c3f48690,e658bcfc,18,c08a5d78,c08d3d98,...) at sendto+0x48
syscall(e658bd38) at syscall+0x274
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (133, FreeBSD ELF32, sendto), eip = 0x2816bc83, esp = 0xbfbfd73c,
ebp = 0xbfbfd768 ---
lock order reversal:
 1st 0xc423f150 tcpinp (tcpinp) @ /usr/src/sys/netinet/tcp_usrreq.c:472
 2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 /usr/src/sys/net/pfil.c:73
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e65a3a30,c05eb7b6,c088f4ad,c09775d8,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
kdb_backtrace+0x29
witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
witness_checkorder+0x6d6
_rw_rlock(c09775d8,c0897d93,49,e65a3af0,c423f0b4,...) at _rw_rlock+0x8e
pfil_run_hooks(c09775c0,e65a3b10,c3d44000,2,c423f0b4,...) at pfil_run_hooks+0x35
ip_output(c3c94e00,0,e65a3ad4,0,0,...) at ip_output+0x90f
tcp_output(c42421d0,c3d2bc50,1d8,c423f150,c4259000,...) at tcp_output+0x140c
tcp_usr_connect(c4259000,c3d2bc50,c3d2f8c0,25,e65a3c64,...) at
tcp_usr_connect+0x11c
soconnect(c4259000,c3d2bc50,c3d2f8c0,10,16,...) at soconnect+0x52
kern_connect(c3d2f8c0,9,c3d2bc50,c3d2bc50,0,...) at kern_connect+0x59
connect(c3d2f8c0,e65a3cfc,c,c088ff65,c08d3a50,...) at connect+0x46
syscall(e65a3d38) at syscall+0x274
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (98, FreeBSD ELF32, connect), eip = 0x28161e9b, esp = 0xbfbfe71c,
ebp = 0xbfbfe868 ---
lock order reversal:
 1st 0xc3eda524 tcp_sc_head (tcp_sc_head) @
 /usr/src/sys/netinet/tcp_syncache.c:494
 2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 /usr/src/sys/net/pfil.c:73
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e4257854,c05eb7b6,c088f4ad,c09775d8,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
kdb_backtrace+0x29
witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
witness_checkorder+0x6d6
_rw_rlock(c09775d8,c0897d93,49,e4257914,0,...) at _rw_rlock+0x8e
pfil_run_hooks(c09775c0,e4257934,c3c31c00,2,0,...) at pfil_run_hooks+0x35
ip_output(c3ef7a00,0,e42578f8,0,0,...) at ip_output+0x90f
syncache_respond(c426ad70,c40c0834,0,0,c40c0834,...) at syncache_respond+0x3a2
_syncache_add(c42400b4,e4257ba8,c40b3700,0,0,...) at _syncache_add+0x2b0
syncache_add(e4257b68,e4257b90,c40c0834,c42400b4,e4257ba8,...) at
syncache_add+0x38
tcp_input(c40b3700,14,c3c31c00,1,0,...) at tcp_input+0xd6b
ip_input(c40b3700,c40b3700,800,c3c31c00,800,...) at ip_input+0x686
netisr_dispatch(2,c40b3700,10,3,0,...) at netisr_dispatch+0x72
ether_demux(c3c31c00,c40b3700,3,0,3,...) at ether_demux+0x2e5
ether_input(c3c31c00,c40b3700,c0aa0a74,6a9,ffffffff,...) at ether_input+0x37f
fxp_intr_body(ffffffff,0,c0aa0a74,5db,c3c33014,...) at fxp_intr_body+0x1c4
fxp_intr(c3c33000,0,c08866ae,4b6,c3b3c268,...) at fxp_intr+0xa0
ithread_loop(c3c1fa50,e4257d38,c0886453,31c,c3bef2b8,...) at ithread_loop+0x1c5
fork_exit(c0590660,c3c1fa50,e4257d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe4257d70, ebp = 0 ---
lock order reversal:
 1st 0xc09786cc udp (udp) @ /usr/src/sys/netinet/udp_usrreq.c:395
 2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 /usr/src/sys/net/pfil.c:73
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e42579b8,c05eb7b6,c088f4ad,c09775d8,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
kdb_backtrace+0x29
witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
witness_checkorder+0x6d6
_rw_rlock(c09775d8,c0897d93,49,e4257a78,0,...) at _rw_rlock+0x8e
pfil_run_hooks(c09775c0,e4257a98,c3c31c00,2,0,...) at pfil_run_hooks+0x35
ip_output(c3efae00,0,e4257a5c,0,0,...) at ip_output+0x90f
icmp_reflect(c40c6020,c3efaec8,14,c3efaf00,c40c6020,...) at icmp_reflect+0x3df
icmp_error(c40b4d00,3,3,0,0,...) at icmp_error+0x3bd
udp_input(c40b4d00,14,c3c31c00,1,0,...) at udp_input+0x5ea
ip_input(c40b4d00,c40b4d00,800,c3c31c00,800,...) at ip_input+0x686
netisr_dispatch(2,c40b4d00,10,3,0,...) at netisr_dispatch+0x72
ether_demux(c3c31c00,c40b4d00,3,0,3,...) at ether_demux+0x2e5
ether_input(c3c31c00,c40b4d00,c0aa0a74,6a9,ffffffff,...) at ether_input+0x37f
fxp_intr_body(ffffffff,0,c0aa0a74,5db,c3c33014,...) at fxp_intr_body+0x1c4
fxp_intr(c3c33000,0,c08866ae,4b6,c3b3c268,...) at fxp_intr+0xa0
ithread_loop(c3c1fa50,e4257d38,c0886453,31c,c3bef2b8,...) at ithread_loop+0x1c5
fork_exit(c0590660,c3c1fa50,e4257d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe4257d70, ebp = 0 ---





kernel backtrace:

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:
panic: _rw_rlock (tcp): wlock already held @
/usr/src/sys/contrib/pf/net/pf.c:3016
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper(c088cf61,e6846220,c05ae7df,c08b659d,0,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c08b659d,0,c0889c7e,e684622c,0,...) at kdb_backtrace+0x29
panic(c0889c7e,c085a754,c088f55e,c087092d,bc8,...) at panic+0x10f
_rw_rlock(c097830c,c087092d,bc8,c08d9624,c087092d,...) at _rw_rlock+0x73
pf_socket_lookup(2,e68463dc,0,cc4,3,...) at pf_socket_lookup+0x208
pf_test_tcp(e6846444,e6846440,2,c3efee00,c3c8e900,...) at pf_test_tcp+0x142
pf_test6(2,c3d44000,e68464a0,0,0,...) at pf_test6+0x8a0
pf_check6_out(0,e68464a0,c3d44000,2,0,...) at pf_check6_out+0x47
pfil_run_hooks(c097ad00,e6846638,c3d44000,2,0,...) at pfil_run_hooks+0x88
ip6_output(c3c8e900,0,e6846618,0,0,...) at ip6_output+0x122e
pf_send_tcp(c4fcfe00,c41259b4,1c,c4fcfe5c,c4fcfe4c,...) at pf_send_tcp+0x6dd
pf_test_tcp(e68468e8,e68468e4,2,c3f20900,c4fcfe00,...) at pf_test_tcp+0xcef
pf_test6(2,c3f06400,e6846944,0,c446b7bc,...) at pf_test6+0x8a0
pf_check6_out(0,e6846944,c3f06400,2,c446b7bc,...) at pf_check6_out+0x47
pfil_run_hooks(c097ad00,e6846adc,c3f06400,2,c446b7bc,...) at pfil_run_hooks+0x88
ip6_output(c4fcfe00,0,e6846abc,0,0,...) at ip6_output+0x122e
tcp_output(c45553a0,c447e7c0,201,c446b858,c45553a0,...) at tcp_output+0x137e
tcp6_usr_connect(c50cd340,c447e7c0,c4eed690,25,e6846c64,...) at
tcp6_usr_connect+0x171
soconnect(c50cd340,c447e7c0,c4eed690,1c,16,...) at soconnect+0x52
kern_connect(c4eed690,3,c447e7c0,c447e7c0,0,...) at kern_connect+0x59
connect(c4eed690,e6846cfc,c,c08a288e,c08d3a50,...) at connect+0x46
syscall(e6846d38) at syscall+0x274
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (98, FreeBSD ELF32, connect), eip = 0x282e6e9b, esp = 0xbfbfe7ec,
ebp = 0xbfbfe848 ---
KDB: enter: panic
shared rw PFil hook read/write mutex r = 1 (0xc097ad18) locked @
/usr/src/sys/net/pfil.c:73
exclusive rw tcpinp r = 0 (0xc446b858) locked @
/usr/src/sys/netinet/tcp_usrreq.c:513
exclusive rw tcp r = 0 (0xc097830c) locked @
/usr/src/sys/netinet/tcp_usrreq.c:510
exclusive sx so_rcv_sx r = 0 (0xc452fbec) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc483cbec) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc4e89bec) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc4e8970c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc483c22c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc480d70c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc4e8a08c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc4e8a56c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc41a456c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc41c156c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc41c18ac) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc41c1bec) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
exclusive sx so_rcv_sx r = 0 (0xc41f108c) locked @
/usr/src/sys/kern/uipc_sockbuf.c:148
shared rw udpinp r = 0 (0xc400f63c) locked @
/usr/src/sys/netinet/udp_usrreq.c:878
Uptime: 16h23m36s
Physical memory: 1015 MB
Dumping 166 MB: 151 135 119 103 87 71 55 39 23 7

Reading symbols from /boot/kernel/if_em.ko...Reading symbols from
/boot/kernel/if_em.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/if_em.ko
Reading symbols from /boot/kernel/if_fxp.ko...Reading symbols from
/boot/kernel/if_fxp.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/if_fxp.ko
Reading symbols from /boot/kernel/miibus.ko...Reading symbols from
/boot/kernel/miibus.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/miibus.ko
Reading symbols from /boot/kernel/if_vr.ko...Reading symbols from
/boot/kernel/if_vr.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/if_vr.ko
Reading symbols from /boot/kernel/ulpt.ko...Reading symbols from
/boot/kernel/ulpt.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ulpt.ko
Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from
/boot/kernel/accf_http.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/accf_http.ko
Reading symbols from /boot/kernel/acpi.ko...Reading symbols from
/boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
#0  doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
    in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:196
#1  0xc05ae54c in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc05ae816 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:572
#3  0xc05acf63 in _rw_rlock (rw=0xc097830c, file=0xc087092d
"/usr/src/sys/contrib/pf/net/pf.c", line=3016)
    at /usr/src/sys/kern/kern_rwlock.c:253
#4  0xc0473e58 in pf_socket_lookup (direction=2, pd=0xe68463dc, inp_arg=0x0) at
/usr/src/sys/contrib/pf/net/pf.c:3016
#5  0xc047dd62 in pf_test_tcp (rm=0xe6846444, sm=0xe6846440, direction=2,
kif=0xc3efee00, m=0xc3c8e900, off=40, 
    h=0xc3c8e944, pd=0xe68463dc, am=0xe6846448, rsm=0xe684643c, ifq=0x0,
    inp=0x0)
    at /usr/src/sys/contrib/pf/net/pf.c:3270
#6  0xc04816c0 in pf_test6 (dir=2, ifp=0xc3d44000, m0=0xe68464a0, eh=0x0,
inp=0x0)
    at /usr/src/sys/contrib/pf/net/pf.c:7368
#7  0xc0484e37 in pf_check6_out (arg=0x0, m=0xe68464a0, ifp=0xc3d44000, dir=2,
inp=0x0)
    at /usr/src/sys/contrib/pf/net/pf_ioctl.c:3739
#8  0xc0657618 in pfil_run_hooks (ph=0xc097ad00, mp=0xe6846638, ifp=0xc3d44000,
dir=2, inp=0x0)
    at /usr/src/sys/net/pfil.c:78
#9  0xc07034fe in ip6_output (m0=0xc3c8e900, opt=0x0, ro=0xe6846618,
flags=Variable "flags" is not available.
) at /usr/src/sys/netinet6/ip6_output.c:853
#10 0xc0477dad in pf_send_tcp (replyto=0xc4fcfe00, r=0xc41259b4, af=28 '\034',
saddr=0xc4fcfe5c, daddr=0xc4fcfe4c, 
    sport=20480, dport=46591, seq=0, ack=1170313007, flags=20 '\024', win=0,
    mss=0, ttl=0 '\0', tag=1, rtag=0, eh=0x0, 
    ifp=0xc3f06400) at /usr/src/sys/contrib/pf/net/pf.c:1978
#11 0xc047e90f in pf_test_tcp (rm=0xe68468e8, sm=0xe68468e4, direction=2,
kif=0xc3f20900, m=0xc4fcfe00, off=40, 
    h=0xc4fcfe44, pd=0xe6846880, am=0xe68468ec, rsm=0xe68468e0, ifq=0x0,
    inp=0xc446b7bc)
    at /usr/src/sys/contrib/pf/net/pf.c:3424
#12 0xc04816c0 in pf_test6 (dir=2, ifp=0xc3f06400, m0=0xe6846944, eh=0x0,
inp=0xc446b7bc)
    at /usr/src/sys/contrib/pf/net/pf.c:7368
#13 0xc0484e37 in pf_check6_out (arg=0x0, m=0xe6846944, ifp=0xc3f06400, dir=2,
inp=0xc446b7bc)
    at /usr/src/sys/contrib/pf/net/pf_ioctl.c:3739
#14 0xc0657618 in pfil_run_hooks (ph=0xc097ad00, mp=0xe6846adc, ifp=0xc3f06400,
dir=2, inp=0xc446b7bc)
    at /usr/src/sys/net/pfil.c:78
#15 0xc07034fe in ip6_output (m0=0xc4fcfe00, opt=0x0, ro=0xe6846abc,
flags=Variable "flags" is not available.
) at /usr/src/sys/netinet6/ip6_output.c:853
#16 0xc06debbe in tcp_output (tp=0xc45553a0) at
/usr/src/sys/netinet/tcp_output.c:1114
#17 0xc06ea5d1 in tcp6_usr_connect (so=0xc50cd340, nam=0xc447e7c0,
td=0xc4eed690) at tcp_offload.h:257
#18 0xc060b002 in soconnect (so=0xc50cd340, nam=0xc447e7c0, td=0xc4eed690) at
/usr/src/sys/kern/uipc_socket.c:771
#19 0xc06129e9 in kern_connect (td=0xc4eed690, fd=3, sa=0xc447e7c0) at
/usr/src/sys/kern/uipc_syscalls.c:570
#20 0xc0612b56 in connect (td=0xc4eed690, uap=0xe6846cfc) at
/usr/src/sys/kern/uipc_syscalls.c:534
#21 0xc083a2d4 in syscall (frame=0xe6846d38) at
/usr/src/sys/i386/i386/trap.c:1090
#22 0xc0821220 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
#23 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

>How-To-Repeat:

>Fix:



>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-pf 
Responsible-Changed-By: remko 
Responsible-Changed-When: Wed Sep 17 13:06:54 UTC 2008 
Responsible-Changed-Why:  
Reassign to PF team 

http://www.freebsd.org/cgi/query-pr.cgi?pr=127439 

From: Geoffrey Mainland <mainland@apeiron.net>
To: Christian Peron <csjp@freebsd.org>
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/127439: deadlock in pf
Date: Wed, 17 Sep 2008 12:21:15 -0400

 Sure, attached below.
 
 ext_if = "fxp0"
 int_if = "em0"
 wifi_if = "vr0"
 vpn_if = "tun0"
 
 rfc1918_nets = "{ 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 }"
 ext_net = "{ 68.164.219.97/24 }"
 int_net = "{ 192.168.0.0/24 }"
 wifi_net = "{ 192.168.1.0/24 }"
 vpn_net = "{ 192.168.2.0/24 }"
 
 ext_zeno = 68.164.219.98
 ext_hamilton = 68.164.219.99
 ext_anaximander = 68.164.219.100
 ext_laplace = 68.164.219.100
 ext_hilbert = 68.164.219.101
 
 ext_nat = $ext_zeno
 
 int_zeno = 192.168.0.10
 int_hamilton = 192.168.0.11
 int_anaximander = 192.168.0.12
 int_laplace = 192.168.0.13
 int_hilbert = 192.168.0.16
 
 int_vince = $int_anaximander
 
 wifi_gateway = 192.168.1.1
 wifi_laplace = 192.168.1.13
 
 icmp_types = "echoreq"
 
 # Supposedly 384Kb up, 1.5Mb down. We set the bandwidth to 300Kbps to get the
 # best performance out of the TCP ACK queue.
 upstream = 300Kb
 downstream = 1.5Mb
 
 #
 # Common ports
 #
 ssh_ports = "{ ssh }"
 http_ports = "{ http, https }"
 vpn_ports = "{ 1194 }"
 mysqld_ports = "{ 3306 }"
 
 # AIM: 5190
 # MSN: 1863, 6891-6900 for file transfers
 # Yahoo: 5050, webcam 5100
 # Jabber: 5222, 5269
 aim_ports = "{ 5190 }"
 yahoo_ports = "{ 5050, 5100 }"
 msn_ports = "{ 1863 }"
 
 emule_tcp_ports = "{ 4662 }"
 emule_udp_ports = "{ 4662, 4665, 4672 }"
 
 bittorrent_ports = "{ 3724, 6112, 6881:6999, 46300:46400}"
 realplayer_ports = "{ 7070 }"
 battlenet_ports = "{ 6112:6119 }"
 nwn_ports = "{ 1070:3000, 5120:5300, 6500, 27900, 28900 }"
 gamespy_ports = "{ 6667, 3783, 27900, 28900, 29900, 29901, 13139, 6515, 6500, 6501 }"
 directx_ports = "{ 47624, 6073, 2300:2400 }"
 ts_tcp_ports = "{ 14534, 51234 }"
 ts_udp_ports = "{ 8767:8768 }"
 
 ################################################################################
 # Options
 #
 ################################################################################
 set block-policy return
 set loginterface $ext_if
 
 ################################################################################
 # Normalization
 #
 ################################################################################
 scrub in all
 
 ################################################################################
 #
 # Queueing
 #
 ################################################################################
 #altq on $ext_if priq bandwidth $upstream queue \
 #  { std_out, im_out, ssh_out, dns_out, tcp_ack_out }
 
 #queue std_out     priq(default)
 #queue im_out      priority 4 priq(red)
 #queue ssh_out     priority 5 priq(red)
 #queue dns_out     priority 6
 #queue tcp_ack_out priority 7
 
 #altq on $int_if cbq bandwidth 100% queue \
 #  { all_in }
 
 #queue all_in bandwidth 100% { int_in, ext_in }
 #  queue int_in bandwidth 8Mb cbq(default)
 #  queue ext_in bandwidth $downstream {std_in, im_in, ssh_in, dns_in, vince_in }
 #    queue std_in   bandwidth 500Kb cbq(borrow)
 #    queue im_in    bandwidth 100Kb priority 4
 #    queue ssh_in   bandwidth 100Kb priority 5
 #    queue dns_in   bandwidth 100Kb priority 6
 #    queue vince_in bandwidth 100Kb cbq(borrow)
 
 ################################################################################
 # Translation
 #
 ################################################################################
 
 # cantor
 rdr pass on $ext_if proto tcp from any to $ext_zeno port 47000:48000 -> 192.168.0.39 port 47000:*
 
 # hamilton
 rdr on $int_if proto tcp from any to $ext_hamilton -> $int_hamilton
 binat on $ext_if from $int_hamilton to any -> $ext_hamilton
 
 # anaximander
 rdr on $int_if proto tcp from any to $ext_anaximander -> $int_anaximander
 binat on $ext_if from $int_anaximander to any -> $ext_anaximander
 
 # laplace
 #rdr on $int_if proto tcp from any to $ext_laplace -> $int_laplace
 #binat on $ext_if from $int_laplace to any -> $ext_laplace
 
 # hilbert
 rdr on $int_if proto tcp from any to $ext_hilbert -> $int_hilbert
 binat on $ext_if from $int_hilbert to any -> $ext_hilbert
 
 nat on $ext_if from $int_if:network -> $ext_nat
 nat on $ext_if from $vpn_net -> $ext_nat
 
 # wifi
 nat on $ext_if from $wifi_if:network -> $ext_nat
 
 # NAT and FTP
 #rdr on $int_if proto tcp from any to any port ftp -> 127.0.0.1 port 8021 
 
 ################################################################################
 # Filtering
 #
 ################################################################################
 
 #
 # Block by default
 #
 block quick on $ext_if proto {udp, tcp} from any to any \
   port { 135, 139, 445 }
 block log all
 
 #
 # Blacklist
 #
 #block quick from 194.139.33.69 to any
 
 #
 # Whitelist
 #
 whitelist = "{ 140.247.60.67 }"
 
 pass quick inet proto tcp from $whitelist to any \
     flags S/SA keep state
 pass quick inet proto udp from $whitelist to any \
     keep state
 
 #
 # Allow anything on the loopback interface
 #
 pass quick on lo0 all
 
 #
 # RFC 1918 addresses should not be seen on the external interface
 #
 block drop in  quick on $ext_if from $rfc1918_nets to any
 block drop out quick on $ext_if from any to $rfc1918_nets
 
 #
 # Protect against spoofing
 #
 antispoof for lo0
 antispoof for $ext_if
 antispoof for $int_if
 antispoof for $wifi_if
 antispoof for $vpn_if
 
 #
 # Ports we open for zeno
 #
 
 # Mail and news
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port { smtp, smtps, submission, imaps, nntps, auth } \
   flags S/SA keep state \
   #queue std_in
 
 # auth
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port { auth } \
   flags S/SA keep state \
   #queue std_in
 
 # HTTP
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port $http_ports \
   flags S/SA keep state \
   #queue std_in
 
 # VPN
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port $vpn_ports \
   flags S/SA keep state \
   #queue std_in
 
 pass in on $ext_if inet proto udp from any to ($ext_if) \
   port $vpn_ports \
   keep state \
   #queue std_in
 
 # SSH
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port $ssh_ports \
   flags S/SA keep state \
   #queue(std_in, ssh_in)
 
 # FTP
 pass in on $ext_if proto tcp from any to ($ext_if) \
   port ftp keep state \
   #queue std_in
 
 pass in on $ext_if proto tcp from any to ($ext_if) \
   port > 49151 keep state \
   #queue std_in
 
 # TeamSpeak
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port $ts_tcp_ports \
   flags S/SA keep state
 
 pass in on $ext_if inet proto udp from any to ($ext_if) \
   port $ts_udp_ports \
   keep state
 
 # DNS
 pass out on $ext_if inet proto { tcp udp } from ($ext_if) to any port domain \
   modulate state \
   #queue dns_out
 
 #
 # Ports we open up for everyone
 #
 
 # ssh
 pass in on $ext_if inet proto tcp from any to $int_net \
   port $ssh_ports \
   flags S/SA keep state
 
 pass out on $ext_if inet proto tcp from ($ext_if) to any \
   port $ssh_ports \
   flags S/SA modulate state \
   #queue(std_out, ssh_out)
 
 # FTP
 pass in on $ext_if inet proto tcp from any to $ext_nat \
   user proxy flags S/SA modulate state
 
 # AIM
 pass in on $ext_if inet proto tcp from any to $int_net \
   port $aim_ports \
   flags S/SA keep state
 
 pass in on $ext_if inet proto udp from any to $int_net \
   port $aim_ports \
   keep state
 
 pass out on $ext_if inet proto tcp from ($ext_if) to any \
   port $aim_ports \
   flags S/SA keep state \
   #queue(im_out, tcp_ack_out)
 
 pass out on $ext_if inet proto udp from ($ext_if) to any \
   port $aim_ports \
   modulate state \
   #queue(im_out)
 
 # Yahoo
 pass in on $ext_if inet proto tcp from any to $int_net \
   port $yahoo_ports \
   flags S/SA keep state
 
 pass in on $ext_if inet proto udp from any to $int_net \
   port $yahoo_ports \
   keep state
 
 pass out on $ext_if inet proto tcp from ($ext_if) to any \
   port $yahoo_ports \
   flags S/SA modulate state \
   #queue(im_out, tcp_ack_out)
 
 # emule
 pass in on $ext_if inet proto tcp from any to $int_net \
   port $emule_tcp_ports \
   flags S/SA keep state
 
 pass in on $ext_if inet proto udp from any to $int_net \
   port $emule_udp_ports \
   modulate state
 
 # BitTorrent
 pass in on $ext_if inet proto tcp from any to $int_net \
   port $bittorrent_ports \
   flags S/SA keep state
 
 pass in on $ext_if inet proto udp from any to $int_net \
   port $bittorrent_ports \
   keep state
 
 # Realplayer
 pass in on $ext_if inet proto udp from any to $int_net \
   port $realplayer_ports \
   keep state
 
 # Battlenet
 pass in on $ext_if inet proto tcp from any to $int_net \
   port $battlenet_ports \
   flags S/SA keep state
 
 # Neverwinter Nights
 #pass in on $ext_if inet proto udp from any to $int_net \
 #  port $nwn_ports \
 #  keep state
 
 # Gamespy Arcade
 #pass in on $ext_if inet proto tcp from any to $int_net \
 #  port $gamespy_ports \
 #  flags S/SA keep state
 
 # DirectX Gaming
 #pass in on $ext_if inet proto tcp from any to $int_net \
 #  port $directx_ports \
 #  flags S/SA keep state
 
 # MySQL
 pass in on $ext_if inet proto tcp from any to ($ext_if) \
   port $mysqld_ports flags S/SA keep state \
 
 #
 # ICMP
 #
 pass in inet proto icmp all icmp-type $icmp_types keep state
 
 #
 # Allow traffic to flow freely between firewall and internal network
 #
 pass in  on $int_if from $int_if:network to any keep state
 pass out on $int_if from any to $int_if:network modulate state
 #pass out on $int_if from any to $int_vince modulate state \
 #  #queue(vince_in)
 
 #
 # Allow traffic to flow freely between firewall and wifi network
 #
 pass in  on $wifi_if from $wifi_if:network to any keep state
 pass out on $wifi_if from any to $wifi_if:network modulate state
 
 #pass in on $wifi_if inet proto udp from $wifi_if:network \
 #  to {$ext_zeno, $wifi_gateway} port 1194 \
 #  keep state
 
 #pass out on $wifi_if inet proto udp from {$ext_zeno, $wifi_gateway} port 1194 \
 #  to $wifi_if:network \
 #  modulate state
 
 #
 # Allow traffic to flow freely between firewall and vpn network
 #
 pass in  on $vpn_if from $vpn_net to any keep state
 pass out on $vpn_if from any to $vpn_net modulate state
 
 #
 # Allow all outgoing traffic from the firewall to the external network
 #
 pass out on $ext_if proto tcp all flags S/SA modulate state \
   #queue(std_out, tcp_ack_out)
 pass out on $ext_if proto { udp, icmp } all keep state
 
 #
 # IPv6
 #
 pass out quick proto ipv6 from any to any keep state
 pass out quick proto ipv6-icmp from any to any keep state
 

From: Christian Peron <csjp@freebsd.org>
To: Geoffrey Mainland <mainland@apeiron.net>
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/127439: deadlock in pf
Date: Wed, 17 Sep 2008 11:16:01 -0500

 Can you provide a copy of your pf ruleset?
 
 On Wed, Sep 17, 2008 at 08:33:23AM -0400, Geoffrey Mainland wrote:
 > 
 > >Number:         127439
 > >Category:       kern
 > >Synopsis:       deadlock in pf
 > >Confidential:   no
 > >Severity:       critical
 > >Priority:       high
 > >Responsible:    freebsd-bugs
 > >State:          open
 > >Quarter:        
 > >Keywords:       
 > >Date-Required:
 > >Class:          sw-bug
 > >Submitter-Id:   current-users
 > >Arrival-Date:   Wed Sep 17 12:50:01 UTC 2008
 > >Closed-Date:
 > >Last-Modified:
 > >Originator:     Geoffrey Mainland
 > >Release:        FreeBSD 7.1-PRERELEASE i386
 > >Organization:
 > >Environment:
 > System: FreeBSD zeno.apeiron.net 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #7: Tue Sep 16 09:28:16 EDT 2008 toor@zeno.apeiron.net:/usr/obj/usr/src/sys/ZENO i386
 > 
 > 
 > >Description:
 > 
 > This happens reliably every night. I'm not sure what's running that triggers it.
 > 
 > ifconfig:
 > 
 > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 >         options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
 >         ether 00:0e:0c:5f:c1:f8
 >         inet6 fe80::20e:cff:fe5f:c1f8%em0 prefixlen 64 scopeid 0x1 
 >         inet 192.168.0.10 netmask 0xffffff00 broadcast 192.168.0.255
 >         inet 192.168.0.1 netmask 0xffffffff broadcast 192.168.0.1
 >         inet 192.168.0.2 netmask 0xffffffff broadcast 192.168.0.2
 >         media: Ethernet autoselect (100baseTX <full-duplex>)
 >         status: active
 > fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 >         options=8<VLAN_MTU>
 >         ether 00:90:27:62:87:4d
 >         inet6 fe80::290:27ff:fe62:874d%fxp0 prefixlen 64 scopeid 0x2 
 >         inet 68.164.219.98 netmask 0xfffffff8 broadcast 68.164.219.103
 >         inet 68.164.219.99 netmask 0xffffffff broadcast 68.164.219.99
 >         inet 68.164.219.100 netmask 0xffffffff broadcast 68.164.219.100
 >         inet 68.164.219.101 netmask 0xffffffff broadcast 68.164.219.101
 >         media: Ethernet autoselect (100baseTX <full-duplex>)
 >         status: active
 > vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 >         options=2808<VLAN_MTU,WOL_UCAST,WOL_MAGIC>
 >         ether 00:15:f2:43:48:7b
 >         inet6 fe80::215:f2ff:fe43:487b%vr0 prefixlen 64 scopeid 0x3 
 >         inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
 >         inet 192.168.1.2 netmask 0xffffffff broadcast 192.168.1.2
 >         media: Ethernet autoselect (none)
 >         status: no carrier
 > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
 >         inet6 ::1 prefixlen 128 
 >         inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 
 >         inet 127.0.0.1 netmask 0xff000000 
 > pfsync0: flags=0<> metric 0 mtu 1460
 >         syncpeer: 224.0.0.240 maxupd: 128
 > pflog0: flags=0<> metric 0 mtu 33204
 > gif0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1280
 >         tunnel inet 68.164.219.98 --> 66.55.128.25
 >         inet6 fe80::20e:cff:fe5f:c1f8%gif0 prefixlen 64 scopeid 0x7 
 >         inet6 2001:4830:1200:10b::2 --> 2001:4830:1200:10b::1 prefixlen 128 
 > tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
 >         inet6 fe80::20e:cff:fe5f:c1f8%tun0 prefixlen 64 scopeid 0x8 
 >         inet 192.168.2.1 --> 192.168.2.2 netmask 0xffffffff 
 >         Opened by PID 1454
 > 
 > Kernel config:
 > 
 > cpu		I686_CPU
 > ident		ZENO
 > options 	SCHED_ULE
 > options		SMP
 > options         PREEMPTION
 > options         DEVICE_POLLING
 > options		HZ=2000
 > options		_KPOSIX_PRIORITY_SCHEDULING
 > options		P1003_1B_MQUEUE
 > options		KDB
 > options		KDB_TRACE
 > options		DDB
 > options		WITNESS
 > options		INVARIANTS
 > options		INVARIANT_SUPPORT
 > makeoptions	DEBUG=-g		#Build kernel with gdb(1) debug symbols
 > options 	COMPAT_FREEBSD4
 > options 	COMPAT_FREEBSD5
 > options 	COMPAT_FREEBSD6
 > options 	SYSVSHM
 > options 	SYSVSEM
 > options 	SYSVMSG
 > options 	STACK
 > options 	INET			#Internet communications protocols
 > options 	INET6			#IPv6 communications protocols
 > options 	IPSEC			#IP security (requires device crypto)
 > options 	NETATALK		#Appletalk communications protocols
 > options 	NETSMB			#SMB/CIFS requester
 > options 	LIBMCHAIN
 > options         SCTP
 > options 	NETGRAPH		# netgraph(4) system
 > device		ether			#Generic Ethernet
 > device		loop			#Network loopback device
 > device		bpf			#Berkeley packet filter
 > device		tap			#Virtual Ethernet driver
 > device		tun			#Tunnel driver (ppp(8), nos-tun(8))
 > device		gre			#IP over IP tunneling
 > device		pf			#PF OpenBSD packet-filter firewall
 > device		pflog			#logging support interface for PF
 > device		pfsync			#synchronization interface for PF
 > device		gif			#IPv6 and IPv4 tunneling
 > device		faith			#for IPv6 and IPv4 translation
 > device		stf			#6to4 IPv6 over IPv4 encapsulation
 > options 	FFS			#Fast filesystem
 > options 	NFSCLIENT		#Network File System client
 > options 	CD9660			#ISO 9660 filesystem
 > options 	MSDOSFS			#MS DOS File System (FAT, FAT32)
 > options 	NFSSERVER		#Network File System server
 > options		NFSLOCKD		#Network Lock Manager
 > options 	NTFS			#NT File System
 > options 	PROCFS			#Process filesystem (requires PSEUDOFS)
 > options 	PSEUDOFS		#Pseudo-filesystem framework
 > options 	SMBFS			#SMB/CIFS filesystem
 > options 	UDF			#Universal Disk Format
 > options 	NFS_ROOT		#NFS usable as root device
 > options 	SOFTUPDATES
 > options 	UFS_ACL
 > options 	UFS_DIRHASH
 > device		random
 > device		mem
 > options 	AUDIT
 > device		scbus		#base SCSI code
 > device		da		#SCSI direct access devices (aka disks)
 > device		cd		#SCSI CD-ROMs
 > device		pt		#SCSI processor
 > device		pass		#CAM passthrough driver
 > device		pty		#Pseudo ttys
 > device		md		#Memory/malloc disk
 > options 	LIBICONV
 > options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
 > device		splash			# Splash screen and screen saver support
 > device		sc
 > options 	SC_DISABLE_KDBKEY	# disable `debug' key
 > device		ata
 > device		atadisk		# ATA disk drives
 > device		ataraid		# ATA RAID drives
 > device		atapicd		# ATAPI CDROM drives
 > device		atapifd		# ATAPI floppy drives
 > device		atapicam	# emulate ATAPI devices as SCSI ditto via CAM
 > options 	ATA_STATIC_ID
 > device		fdc
 > device		sound
 > device		ppc
 > device		ppbus
 > device		lpt
 > device		ppi
 > device		uhci
 > device		ehci
 > device		usb
 > device		crypto		# core crypto support
 > device		cryptodev	# /dev/crypto for access to h/w
 > device		apic			# I/O apic
 > device		nvram		# Access to rtc cmos via /dev/nvram
 > device		sio
 > device		eisa
 > device		pci
 > options 	VESA
 > device		psm
 > device		atkbdc
 > device		atkbd
 > device		vga
 > options 	COMPAT_LINUX
 > options 	COMPAT_AOUT
 > options 	LINPROCFS
 > options		LINSYSFS
 > 
 > 
 > 
 > 
 > 
 > dmesg output (after crash):
 > 
 > Copyright (c) 1992-2008 The FreeBSD Project.
 > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 > 	  The Regents of the University of California. All rights reserved.
 > FreeBSD is a registered trademark of The FreeBSD Foundation.
 > FreeBSD 7.1-PRERELEASE #7: Tue Sep 16 09:28:16 EDT 2008
 >     toor@zeno.apeiron.net:/usr/obj/usr/src/sys/ZENO
 > WARNING: WITNESS option enabled, expect reduced performance.
 > Timecounter "i8254" frequency 1193182 Hz quality 0
 > CPU: AMD Sempron(tm) Processor 3100+ (1800.09-MHz 686-class CPU)
 >   Origin = "AuthenticAMD"  Id = 0x10fc0  Stepping = 0
 >   Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
 >   AMD Features=0xc2500800<SYSCALL,NX,MMX+,FFXSR,3DNow!+,3DNow!>
 >   AMD Features2=0x1<LAHF>
 > real memory  = 1073414144 (1023 MB)
 > avail memory = 1040887808 (992 MB)
 > WITNESS: spin lock cpuset not in order list
 > WITNESS: spin lock intrcnt not in order list
 > netsmb_dev: loaded
 > cryptosoft0: <software crypto> on motherboard
 > acpi0: <A M I OEMRSDT> on motherboard
 > acpi0: [ITHREAD]
 > acpi0: Power Button (fixed)
 > acpi0: reservation of 0, a0000 (3) failed
 > acpi0: reservation of 100000, 3fef0000 (3) failed
 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
 > pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 > pci0: <ACPI PCI bus> on pcib0
 > pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
 > pci1: <ACPI PCI bus> on pcib1
 > vgapci0: <VGA-compatible display> mem
 > 0xfb000000-0xfbffffff,0xf0000000-0xf7ffffff irq 11 at device 0.0 on pci1
 > em0: <Intel(R) PRO/1000 Network Connection 6.9.5> port 0xe800-0xe83f mem
 > 0xfae00000-0xfae1ffff,0xfad00000-0xfad1ffff irq 11 at device 11.0 on pci0
 > em0: [FILTER]
 > em0: Ethernet address: 00:0e:0c:5f:c1:f8
 > fxp0: <Intel 82559 Pro/100 Ethernet> port 0xe400-0xe43f mem
 > 0xfab00000-0xfab00fff,0xfaa00000-0xfaafffff irq 10 at device 12.0 on pci0
 > miibus0: <MII bus> on fxp0
 > inphy0: <i82555 10/100 media interface> PHY 1 on miibus0
 > inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 > fxp0: Ethernet address: 00:90:27:62:87:4d
 > fxp0: [ITHREAD]
 > atapci0: <VIA 6420 SATA150 controller> port
 > 0xe000-0xe007,0xd800-0xd803,0xd400-0xd407,0xd000-0xd003,0xc800-0xc80f,0xc400-0xc4ff
 > irq 10 at device 15.0 on pci0
 > atapci0: [ITHREAD]
 > ata2: <ATA channel 0> on atapci0
 > ata2: [ITHREAD]
 > ata3: <ATA channel 1> on atapci0
 > ata3: [ITHREAD]
 > atapci1: <VIA 8237 UDMA133 controller> port
 > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 15.1 on pci0
 > ata0: <ATA channel 0> on atapci1
 > ata0: [ITHREAD]
 > ata1: <ATA channel 1> on atapci1
 > ata1: [ITHREAD]
 > uhci0: <VIA 83C572 USB controller> port 0xb000-0xb01f irq 11 at device 16.0 on
 > pci0
 > uhci0: [GIANT-LOCKED]
 > uhci0: [ITHREAD]
 > usb0: <VIA 83C572 USB controller> on uhci0
 > usb0: USB revision 1.0
 > uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
 > uhub0: 2 ports with 2 removable, self powered
 > uhci1: <VIA 83C572 USB controller> port 0xb400-0xb41f irq 11 at device 16.1 on
 > pci0
 > uhci1: [GIANT-LOCKED]
 > uhci1: [ITHREAD]
 > usb1: <VIA 83C572 USB controller> on uhci1
 > usb1: USB revision 1.0
 > uhub1: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
 > uhub1: 2 ports with 2 removable, self powered
 > uhci2: <VIA 83C572 USB controller> port 0xb800-0xb81f irq 10 at device 16.2 on
 > pci0
 > uhci2: [GIANT-LOCKED]
 > uhci2: [ITHREAD]
 > usb2: <VIA 83C572 USB controller> on uhci2
 > usb2: USB revision 1.0
 > uhub2: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
 > uhub2: 2 ports with 2 removable, self powered
 > uhci3: <VIA 83C572 USB controller> port 0xc000-0xc01f irq 10 at device 16.3 on
 > pci0
 > uhci3: [GIANT-LOCKED]
 > uhci3: [ITHREAD]
 > usb3: <VIA 83C572 USB controller> on uhci3
 > usb3: USB revision 1.0
 > uhub3: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
 > uhub3: 2 ports with 2 removable, self powered
 > ehci0: <VIA VT6202 USB 2.0 controller> mem 0xfa700000-0xfa7000ff irq 5 at device
 > 16.4 on pci0
 > ehci0: [GIANT-LOCKED]
 > ehci0: [ITHREAD]
 > usb4: EHCI version 1.0
 > usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
 > usb4: <VIA VT6202 USB 2.0 controller> on ehci0
 > usb4: USB revision 2.0
 > uhub4: <VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
 > uhub4: 8 ports with 8 removable, self powered
 > isab0: <PCI-ISA bridge> at device 17.0 on pci0
 > isa0: <ISA bus> on isab0
 > pci0: <multimedia, audio> at device 17.5 (no driver attached)
 > vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0xa400-0xa4ff mem
 > 0xfa600000-0xfa6000ff irq 11 at device 18.0 on pci0
 > vr0: Quirks: 0x0
 > vr0: Revision: 0x78
 > miibus1: <MII bus> on vr0
 > rlphy0: <RTL8201L 10/100 media interface> PHY 1 on miibus1
 > rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 > vr0: Ethernet address: 00:15:f2:43:48:7b
 > vr0: [ITHREAD]
 > cpu0: <ACPI CPU> on acpi0
 > acpi_button0: <Power Button> on acpi0
 > acpi_button1: <Sleep Button> on acpi0
 > atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
 > atkbd0: <AT Keyboard> irq 1 on atkbdc0
 > kbd0 at atkbd0
 > atkbd0: [GIANT-LOCKED]
 > atkbd0: [ITHREAD]
 > fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
 > acpi0
 > fdc0: [FILTER]
 > fd0: <1440-KB 3.5" drive> on fdc0 drive 0
 > sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 > sio0: type 16550A
 > sio0: [FILTER]
 > orm0: <ISA Option ROMs> at iomem 0xcd000-0xcdfff,0xce000-0xcefff,0xcf000-0xd3fff
 > pnpid ORM0000 on isa0
 > sc0: <System console> at flags 0x100 on isa0
 > sc0: VGA <16 virtual consoles, flags=0x300>
 > vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 > ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
 > ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
 > ppc0: FIFO with 16/16/9 bytes threshold
 > ppbus0: <Parallel port bus> on ppc0
 > ppbus0: [ITHREAD]
 > lpt0: <Printer> on ppbus0
 > lpt0: Interrupt-driven port
 > ppi0: <Parallel I/O> on ppbus0
 > ppc0: [GIANT-LOCKED]
 > ppc0: [ITHREAD]
 > sio1: configured irq 3 not in bitmap of probed irqs 0
 > sio1: port may not be enabled
 > Timecounter "TSC" frequency 1800086355 Hz quality 800
 > Timecounters tick every 1.000 msec
 > IPsec: Initialized Security Association Processing.
 > ad0: 194481MB <Maxtor 6B200P0 BAH41BM0> at ata0-master UDMA133
 > acd0: DVDR <NEC DVD RW ND-3550A/1.05> at ata1-master UDMA33
 > ad4: 239372MB <Maxtor 7L250S0 BANC1G10> at ata2-master SATA150
 > cd0 at ata1 bus 0 target 0 lun 0
 > cd0: <_NEC DVD_RW ND-3550A 1.05> Removable CD-ROM SCSI-0 device 
 > cd0: 33.000MB/s transfers
 > cd0: Attempt to query device size failed: NOT READY, Medium not present
 > WARNING: WITNESS option enabled, expect reduced performance.
 > Trying to mount root from ufs:/dev/ad4s1a
 > WARNING: / was not properly dismounted
 > lock order reversal:
 >  1st 0xc0907fcc pf task mtx (pf task mtx) @
 >  /usr/src/sys/contrib/pf/net/pf_ioctl.c:1394
 >  2nd 0xc0973488 ifnet (ifnet) @ /usr/src/sys/net/if.c:1558
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e658ba3c,c05eb7b6,c088f4ad,c0973488,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c088f4ad,c0973488,c0896cfd,c0896cfd,c0896b56,...) at
 > kdb_backtrace+0x29
 > witness_checkorder(c0973488,9,c0896b56,616,0,...) at witness_checkorder+0x6d6
 > _mtx_lock_flags(c0973488,0,c0896b56,616,c3f37a70,...) at _mtx_lock_flags+0xbc
 > ifunit(c3f37a70,0,c08711f2,572,c05e958e,...) at ifunit+0x2f
 > pfioctl(c3d2d800,c0104414,c3f37a70,3,c3f48690,...) at pfioctl+0x23b5
 > devfs_ioctl_f(c3f49c2c,c0104414,c3f37a70,c3b2c000,c3f48690,...) at
 > devfs_ioctl_f+0xe5
 > kern_ioctl(c3f48690,3,c0104414,c3f37a70,1000000,...) at kern_ioctl+0x243
 > ioctl(c3f48690,e658bcfc,c,c08bade8,c08d3630,...) at ioctl+0x134
 > syscall(e658bd38) at syscall+0x274
 > Xint0x80_syscall() at Xint0x80_syscall+0x20
 > --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x281aac4b, esp = 0xbfbfde5c, ebp
 > = 0xbfbfde88 ---
 > lock order reversal:
 >  1st 0xc097830c tcp (tcp) @ /usr/src/sys/netinet/tcp_input.c:400
 >  2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 >  /usr/src/sys/net/pfil.c:73
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e42579ac,c05eb7b6,c088f4ad,c09775d8,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
 > kdb_backtrace+0x29
 > witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
 > witness_checkorder+0x6d6
 > _rw_rlock(c09775d8,c0897d93,49,e4257a6c,0,...) at _rw_rlock+0x8e
 > pfil_run_hooks(c09775c0,e4257a8c,c3c31c00,2,0,...) at pfil_run_hooks+0x35
 > ip_output(c3c46100,0,e4257a50,0,0,0,c08e7c90,0,0,0,c067c807,c08e7c94,c08e7c9c,c8)
 > at ip_output+0x90f
 > tcp_respond(0,c3c87020,c3c87034,c3c46100,2da9088c,...) at tcp_respond+0x3e7
 > tcp_dropwithreset(1,3,c089c953,353,1900,...) at tcp_dropwithreset+0x152
 > tcp_input(c3c46100,14,c3c31c00,1,0,...) at tcp_input+0xe45
 > ip_input(c3c46100,c3c46100,800,c3c31c00,800,...) at ip_input+0x686
 > netisr_dispatch(2,c3c46100,10,3,0,...) at netisr_dispatch+0x72
 > ether_demux(c3c31c00,c3c46100,3,0,3,...) at ether_demux+0x2e5
 > ether_input(c3c31c00,c3c46100,c0aa0a74,6a9,ffffffff,...) at ether_input+0x37f
 > fxp_intr_body(ffffffff,0,c0aa0a74,5db,c3c33014,...) at fxp_intr_body+0x1c4
 > fxp_intr(c3c33000,0,c08866ae,4b6,c3b3c268,...) at fxp_intr+0xa0
 > ithread_loop(c3c1fa50,e4257d38,c0886453,31c,c3bef2b8,...) at ithread_loop+0x1c5
 > fork_exit(c0590660,c3c1fa50,e4257d38) at fork_exit+0xb8
 > fork_trampoline() at fork_trampoline+0x8
 > --- trap 0, eip = 0, esp = 0xe4257d70, ebp = 0 ---
 > lock order reversal:
 >  1st 0xc4013d44 udpinp (udpinp) @ /usr/src/sys/netinet/udp_usrreq.c:878
 >  2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 >  /usr/src/sys/net/pfil.c:73
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e658ba14,c05eb7b6,c088f4ad,c09775d8,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
 > kdb_backtrace+0x29
 > witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
 > witness_checkorder+0x6d6
 > _rw_rlock(c09775d8,c0897d93,49,e658bad4,c4013ca8,...) at _rw_rlock+0x8e
 > pfil_run_hooks(c09775c0,e658baf4,c3d44000,2,c4013ca8,...) at pfil_run_hooks+0x35
 > ip_output(c3ef6100,0,e658bab8,0,0,...) at ip_output+0x90f
 > udp_send(c42454e0,0,c3ef6100,0,0,...) at udp_send+0x8cd
 > sosend_dgram(c42454e0,0,e658bbec,c3ef6100,0,...) at sosend_dgram+0x351
 > sosend(c42454e0,0,e658bbec,0,0,...) at sosend+0x54
 > kern_sendit(c3f48690,4,e658bc68,0,0,...) at kern_sendit+0xdb
 > sendit(0,8143023,0,0,0,...) at sendit+0xb1
 > sendto(c3f48690,e658bcfc,18,c08a5d78,c08d3d98,...) at sendto+0x48
 > syscall(e658bd38) at syscall+0x274
 > Xint0x80_syscall() at Xint0x80_syscall+0x20
 > --- syscall (133, FreeBSD ELF32, sendto), eip = 0x2816bc83, esp = 0xbfbfd73c,
 > ebp = 0xbfbfd768 ---
 > lock order reversal:
 >  1st 0xc423f150 tcpinp (tcpinp) @ /usr/src/sys/netinet/tcp_usrreq.c:472
 >  2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 >  /usr/src/sys/net/pfil.c:73
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e65a3a30,c05eb7b6,c088f4ad,c09775d8,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
 > kdb_backtrace+0x29
 > witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
 > witness_checkorder+0x6d6
 > _rw_rlock(c09775d8,c0897d93,49,e65a3af0,c423f0b4,...) at _rw_rlock+0x8e
 > pfil_run_hooks(c09775c0,e65a3b10,c3d44000,2,c423f0b4,...) at pfil_run_hooks+0x35
 > ip_output(c3c94e00,0,e65a3ad4,0,0,...) at ip_output+0x90f
 > tcp_output(c42421d0,c3d2bc50,1d8,c423f150,c4259000,...) at tcp_output+0x140c
 > tcp_usr_connect(c4259000,c3d2bc50,c3d2f8c0,25,e65a3c64,...) at
 > tcp_usr_connect+0x11c
 > soconnect(c4259000,c3d2bc50,c3d2f8c0,10,16,...) at soconnect+0x52
 > kern_connect(c3d2f8c0,9,c3d2bc50,c3d2bc50,0,...) at kern_connect+0x59
 > connect(c3d2f8c0,e65a3cfc,c,c088ff65,c08d3a50,...) at connect+0x46
 > syscall(e65a3d38) at syscall+0x274
 > Xint0x80_syscall() at Xint0x80_syscall+0x20
 > --- syscall (98, FreeBSD ELF32, connect), eip = 0x28161e9b, esp = 0xbfbfe71c,
 > ebp = 0xbfbfe868 ---
 > lock order reversal:
 >  1st 0xc3eda524 tcp_sc_head (tcp_sc_head) @
 >  /usr/src/sys/netinet/tcp_syncache.c:494
 >  2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 >  /usr/src/sys/net/pfil.c:73
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e4257854,c05eb7b6,c088f4ad,c09775d8,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
 > kdb_backtrace+0x29
 > witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
 > witness_checkorder+0x6d6
 > _rw_rlock(c09775d8,c0897d93,49,e4257914,0,...) at _rw_rlock+0x8e
 > pfil_run_hooks(c09775c0,e4257934,c3c31c00,2,0,...) at pfil_run_hooks+0x35
 > ip_output(c3ef7a00,0,e42578f8,0,0,...) at ip_output+0x90f
 > syncache_respond(c426ad70,c40c0834,0,0,c40c0834,...) at syncache_respond+0x3a2
 > _syncache_add(c42400b4,e4257ba8,c40b3700,0,0,...) at _syncache_add+0x2b0
 > syncache_add(e4257b68,e4257b90,c40c0834,c42400b4,e4257ba8,...) at
 > syncache_add+0x38
 > tcp_input(c40b3700,14,c3c31c00,1,0,...) at tcp_input+0xd6b
 > ip_input(c40b3700,c40b3700,800,c3c31c00,800,...) at ip_input+0x686
 > netisr_dispatch(2,c40b3700,10,3,0,...) at netisr_dispatch+0x72
 > ether_demux(c3c31c00,c40b3700,3,0,3,...) at ether_demux+0x2e5
 > ether_input(c3c31c00,c40b3700,c0aa0a74,6a9,ffffffff,...) at ether_input+0x37f
 > fxp_intr_body(ffffffff,0,c0aa0a74,5db,c3c33014,...) at fxp_intr_body+0x1c4
 > fxp_intr(c3c33000,0,c08866ae,4b6,c3b3c268,...) at fxp_intr+0xa0
 > ithread_loop(c3c1fa50,e4257d38,c0886453,31c,c3bef2b8,...) at ithread_loop+0x1c5
 > fork_exit(c0590660,c3c1fa50,e4257d38) at fork_exit+0xb8
 > fork_trampoline() at fork_trampoline+0x8
 > --- trap 0, eip = 0, esp = 0xe4257d70, ebp = 0 ---
 > lock order reversal:
 >  1st 0xc09786cc udp (udp) @ /usr/src/sys/netinet/udp_usrreq.c:395
 >  2nd 0xc09775d8 PFil hook read/write mutex (PFil hook read/write mutex) @
 >  /usr/src/sys/net/pfil.c:73
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e42579b8,c05eb7b6,c088f4ad,c09775d8,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c088f4ad,c09775d8,c0897dab,c0897dab,c0897d93,...) at
 > kdb_backtrace+0x29
 > witness_checkorder(c09775d8,1,c0897d93,49,c08a1d09,...) at
 > witness_checkorder+0x6d6
 > _rw_rlock(c09775d8,c0897d93,49,e4257a78,0,...) at _rw_rlock+0x8e
 > pfil_run_hooks(c09775c0,e4257a98,c3c31c00,2,0,...) at pfil_run_hooks+0x35
 > ip_output(c3efae00,0,e4257a5c,0,0,...) at ip_output+0x90f
 > icmp_reflect(c40c6020,c3efaec8,14,c3efaf00,c40c6020,...) at icmp_reflect+0x3df
 > icmp_error(c40b4d00,3,3,0,0,...) at icmp_error+0x3bd
 > udp_input(c40b4d00,14,c3c31c00,1,0,...) at udp_input+0x5ea
 > ip_input(c40b4d00,c40b4d00,800,c3c31c00,800,...) at ip_input+0x686
 > netisr_dispatch(2,c40b4d00,10,3,0,...) at netisr_dispatch+0x72
 > ether_demux(c3c31c00,c40b4d00,3,0,3,...) at ether_demux+0x2e5
 > ether_input(c3c31c00,c40b4d00,c0aa0a74,6a9,ffffffff,...) at ether_input+0x37f
 > fxp_intr_body(ffffffff,0,c0aa0a74,5db,c3c33014,...) at fxp_intr_body+0x1c4
 > fxp_intr(c3c33000,0,c08866ae,4b6,c3b3c268,...) at fxp_intr+0xa0
 > ithread_loop(c3c1fa50,e4257d38,c0886453,31c,c3bef2b8,...) at ithread_loop+0x1c5
 > fork_exit(c0590660,c3c1fa50,e4257d38) at fork_exit+0xb8
 > fork_trampoline() at fork_trampoline+0x8
 > --- trap 0, eip = 0, esp = 0xe4257d70, ebp = 0 ---
 > 
 > 
 > 
 > 
 > 
 > kernel backtrace:
 > 
 > GNU gdb 6.1.1 [FreeBSD]
 > Copyright 2004 Free Software Foundation, Inc.
 > GDB is free software, covered by the GNU General Public License, and you are
 > welcome to change it and/or distribute copies of it under certain conditions.
 > Type "show copying" to see the conditions.
 > There is absolutely no warranty for GDB.  Type "show warranty" for details.
 > This GDB was configured as "i386-marcel-freebsd"...
 > 
 > Unread portion of the kernel message buffer:
 > panic: _rw_rlock (tcp): wlock already held @
 > /usr/src/sys/contrib/pf/net/pf.c:3016
 > cpuid = 0
 > KDB: stack backtrace:
 > db_trace_self_wrapper(c088cf61,e6846220,c05ae7df,c08b659d,0,...) at
 > db_trace_self_wrapper+0x26
 > kdb_backtrace(c08b659d,0,c0889c7e,e684622c,0,...) at kdb_backtrace+0x29
 > panic(c0889c7e,c085a754,c088f55e,c087092d,bc8,...) at panic+0x10f
 > _rw_rlock(c097830c,c087092d,bc8,c08d9624,c087092d,...) at _rw_rlock+0x73
 > pf_socket_lookup(2,e68463dc,0,cc4,3,...) at pf_socket_lookup+0x208
 > pf_test_tcp(e6846444,e6846440,2,c3efee00,c3c8e900,...) at pf_test_tcp+0x142
 > pf_test6(2,c3d44000,e68464a0,0,0,...) at pf_test6+0x8a0
 > pf_check6_out(0,e68464a0,c3d44000,2,0,...) at pf_check6_out+0x47
 > pfil_run_hooks(c097ad00,e6846638,c3d44000,2,0,...) at pfil_run_hooks+0x88
 > ip6_output(c3c8e900,0,e6846618,0,0,...) at ip6_output+0x122e
 > pf_send_tcp(c4fcfe00,c41259b4,1c,c4fcfe5c,c4fcfe4c,...) at pf_send_tcp+0x6dd
 > pf_test_tcp(e68468e8,e68468e4,2,c3f20900,c4fcfe00,...) at pf_test_tcp+0xcef
 > pf_test6(2,c3f06400,e6846944,0,c446b7bc,...) at pf_test6+0x8a0
 > pf_check6_out(0,e6846944,c3f06400,2,c446b7bc,...) at pf_check6_out+0x47
 > pfil_run_hooks(c097ad00,e6846adc,c3f06400,2,c446b7bc,...) at pfil_run_hooks+0x88
 > ip6_output(c4fcfe00,0,e6846abc,0,0,...) at ip6_output+0x122e
 > tcp_output(c45553a0,c447e7c0,201,c446b858,c45553a0,...) at tcp_output+0x137e
 > tcp6_usr_connect(c50cd340,c447e7c0,c4eed690,25,e6846c64,...) at
 > tcp6_usr_connect+0x171
 > soconnect(c50cd340,c447e7c0,c4eed690,1c,16,...) at soconnect+0x52
 > kern_connect(c4eed690,3,c447e7c0,c447e7c0,0,...) at kern_connect+0x59
 > connect(c4eed690,e6846cfc,c,c08a288e,c08d3a50,...) at connect+0x46
 > syscall(e6846d38) at syscall+0x274
 > Xint0x80_syscall() at Xint0x80_syscall+0x20
 > --- syscall (98, FreeBSD ELF32, connect), eip = 0x282e6e9b, esp = 0xbfbfe7ec,
 > ebp = 0xbfbfe848 ---
 > KDB: enter: panic
 > shared rw PFil hook read/write mutex r = 1 (0xc097ad18) locked @
 > /usr/src/sys/net/pfil.c:73
 > exclusive rw tcpinp r = 0 (0xc446b858) locked @
 > /usr/src/sys/netinet/tcp_usrreq.c:513
 > exclusive rw tcp r = 0 (0xc097830c) locked @
 > /usr/src/sys/netinet/tcp_usrreq.c:510
 > exclusive sx so_rcv_sx r = 0 (0xc452fbec) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc483cbec) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc4e89bec) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc4e8970c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc483c22c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc480d70c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc4e8a08c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc4e8a56c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc41a456c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc41c156c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc41c18ac) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc41c1bec) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > exclusive sx so_rcv_sx r = 0 (0xc41f108c) locked @
 > /usr/src/sys/kern/uipc_sockbuf.c:148
 > shared rw udpinp r = 0 (0xc400f63c) locked @
 > /usr/src/sys/netinet/udp_usrreq.c:878
 > Uptime: 16h23m36s
 > Physical memory: 1015 MB
 > Dumping 166 MB: 151 135 119 103 87 71 55 39 23 7
 > 
 > Reading symbols from /boot/kernel/if_em.ko...Reading symbols from
 > /boot/kernel/if_em.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/if_em.ko
 > Reading symbols from /boot/kernel/if_fxp.ko...Reading symbols from
 > /boot/kernel/if_fxp.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/if_fxp.ko
 > Reading symbols from /boot/kernel/miibus.ko...Reading symbols from
 > /boot/kernel/miibus.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/miibus.ko
 > Reading symbols from /boot/kernel/if_vr.ko...Reading symbols from
 > /boot/kernel/if_vr.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/if_vr.ko
 > Reading symbols from /boot/kernel/ulpt.ko...Reading symbols from
 > /boot/kernel/ulpt.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/ulpt.ko
 > Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from
 > /boot/kernel/accf_http.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/accf_http.ko
 > Reading symbols from /boot/kernel/acpi.ko...Reading symbols from
 > /boot/kernel/acpi.ko.symbols...done.
 > done.
 > Loaded symbols for /boot/kernel/acpi.ko
 > #0  doadump () at pcpu.h:196
 > 196 pcpu.h: No such file or directory.
 >     in pcpu.h
 > (kgdb) bt
 > #0  doadump () at pcpu.h:196
 > #1  0xc05ae54c in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
 > #2  0xc05ae816 in panic (fmt=Variable "fmt" is not available.
 > ) at /usr/src/sys/kern/kern_shutdown.c:572
 > #3  0xc05acf63 in _rw_rlock (rw=0xc097830c, file=0xc087092d
 > "/usr/src/sys/contrib/pf/net/pf.c", line=3016)
 >     at /usr/src/sys/kern/kern_rwlock.c:253
 > #4  0xc0473e58 in pf_socket_lookup (direction=2, pd=0xe68463dc, inp_arg=0x0) at
 > /usr/src/sys/contrib/pf/net/pf.c:3016
 > #5  0xc047dd62 in pf_test_tcp (rm=0xe6846444, sm=0xe6846440, direction=2,
 > kif=0xc3efee00, m=0xc3c8e900, off=40, 
 >     h=0xc3c8e944, pd=0xe68463dc, am=0xe6846448, rsm=0xe684643c, ifq=0x0,
 >     inp=0x0)
 >     at /usr/src/sys/contrib/pf/net/pf.c:3270
 > #6  0xc04816c0 in pf_test6 (dir=2, ifp=0xc3d44000, m0=0xe68464a0, eh=0x0,
 > inp=0x0)
 >     at /usr/src/sys/contrib/pf/net/pf.c:7368
 > #7  0xc0484e37 in pf_check6_out (arg=0x0, m=0xe68464a0, ifp=0xc3d44000, dir=2,
 > inp=0x0)
 >     at /usr/src/sys/contrib/pf/net/pf_ioctl.c:3739
 > #8  0xc0657618 in pfil_run_hooks (ph=0xc097ad00, mp=0xe6846638, ifp=0xc3d44000,
 > dir=2, inp=0x0)
 >     at /usr/src/sys/net/pfil.c:78
 > #9  0xc07034fe in ip6_output (m0=0xc3c8e900, opt=0x0, ro=0xe6846618,
 > flags=Variable "flags" is not available.
 > ) at /usr/src/sys/netinet6/ip6_output.c:853
 > #10 0xc0477dad in pf_send_tcp (replyto=0xc4fcfe00, r=0xc41259b4, af=28 '\034',
 > saddr=0xc4fcfe5c, daddr=0xc4fcfe4c, 
 >     sport=20480, dport=46591, seq=0, ack=1170313007, flags=20 '\024', win=0,
 >     mss=0, ttl=0 '\0', tag=1, rtag=0, eh=0x0, 
 >     ifp=0xc3f06400) at /usr/src/sys/contrib/pf/net/pf.c:1978
 > #11 0xc047e90f in pf_test_tcp (rm=0xe68468e8, sm=0xe68468e4, direction=2,
 > kif=0xc3f20900, m=0xc4fcfe00, off=40, 
 >     h=0xc4fcfe44, pd=0xe6846880, am=0xe68468ec, rsm=0xe68468e0, ifq=0x0,
 >     inp=0xc446b7bc)
 >     at /usr/src/sys/contrib/pf/net/pf.c:3424
 > #12 0xc04816c0 in pf_test6 (dir=2, ifp=0xc3f06400, m0=0xe6846944, eh=0x0,
 > inp=0xc446b7bc)
 >     at /usr/src/sys/contrib/pf/net/pf.c:7368
 > #13 0xc0484e37 in pf_check6_out (arg=0x0, m=0xe6846944, ifp=0xc3f06400, dir=2,
 > inp=0xc446b7bc)
 >     at /usr/src/sys/contrib/pf/net/pf_ioctl.c:3739
 > #14 0xc0657618 in pfil_run_hooks (ph=0xc097ad00, mp=0xe6846adc, ifp=0xc3f06400,
 > dir=2, inp=0xc446b7bc)
 >     at /usr/src/sys/net/pfil.c:78
 > #15 0xc07034fe in ip6_output (m0=0xc4fcfe00, opt=0x0, ro=0xe6846abc,
 > flags=Variable "flags" is not available.
 > ) at /usr/src/sys/netinet6/ip6_output.c:853
 > #16 0xc06debbe in tcp_output (tp=0xc45553a0) at
 > /usr/src/sys/netinet/tcp_output.c:1114
 > #17 0xc06ea5d1 in tcp6_usr_connect (so=0xc50cd340, nam=0xc447e7c0,
 > td=0xc4eed690) at tcp_offload.h:257
 > #18 0xc060b002 in soconnect (so=0xc50cd340, nam=0xc447e7c0, td=0xc4eed690) at
 > /usr/src/sys/kern/uipc_socket.c:771
 > #19 0xc06129e9 in kern_connect (td=0xc4eed690, fd=3, sa=0xc447e7c0) at
 > /usr/src/sys/kern/uipc_syscalls.c:570
 > #20 0xc0612b56 in connect (td=0xc4eed690, uap=0xe6846cfc) at
 > /usr/src/sys/kern/uipc_syscalls.c:534
 > #21 0xc083a2d4 in syscall (frame=0xe6846d38) at
 > /usr/src/sys/i386/i386/trap.c:1090
 > #22 0xc0821220 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
 > #23 0x00000033 in ?? ()
 > Previous frame inner to this frame (corrupt stack?)
 > (kgdb) 
 > 
 > >How-To-Repeat:
 > 
 > >Fix:
 > 
 > 
 > 
 > >Release-Note:
 > >Audit-Trail:
 > >Unformatted:
 > _______________________________________________
 > freebsd-bugs@freebsd.org mailing list
 > http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
 > To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@freebsd.org"

From: Christian Peron <csjp@freebsd.org>
To: Geoffrey Mainland <mainland@apeiron.net>
Cc: Christian Peron <csjp@freebsd.org>, FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/127439: deadlock in pf
Date: Wed, 17 Sep 2008 11:47:13 -0500

 On Wed, Sep 17, 2008 at 12:21:15PM -0400, Geoffrey Mainland wrote:
 [..]
 > 
 > # FTP
 > pass in on $ext_if inet proto tcp from any to $ext_nat \
 >   user proxy flags S/SA modulate state
 > 
 
 What happens if you get rid of the "user proxy" constraint?  We have
 had problems with these rules in the past.  The truth is, they don't
 really work correctly anyway.  But it would be interesting to see if
 removing the "user proxy" constraint and replacing it with a port or
 range removes the dead lock.
 

From: Christian Peron <csjp@freebsd.org>
To: Christian Peron <csjp@freebsd.org>
Cc: Geoffrey Mainland <mainland@apeiron.net>, FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/127439: deadlock in pf
Date: Wed, 17 Sep 2008 12:27:43 -0500

 Actually -- ignore this request. This is not the problem.
 
 On Wed, Sep 17, 2008 at 11:47:13AM -0500, Christian Peron wrote:
 > On Wed, Sep 17, 2008 at 12:21:15PM -0400, Geoffrey Mainland wrote:
 > [..]
 > > 
 > > # FTP
 > > pass in on $ext_if inet proto tcp from any to $ext_nat \
 > >   user proxy flags S/SA modulate state
 > > 
 > 
 > What happens if you get rid of the "user proxy" constraint?  We have
 > had problems with these rules in the past.  The truth is, they don't
 > really work correctly anyway.  But it would be interesting to see if
 > removing the "user proxy" constraint and replacing it with a port or
 > range removes the dead lock.
 > 

From: Philippe Pepiot <phil@philpep.org>
To: bug-followup@FreeBSD.org
Cc: mainland@apeiron.net, csjp@freebsd.org
Subject: Re: kern/127439: [pf] deadlock in pf
Date: Sat, 11 Sep 2010 16:04:31 +0200

 Hi, this lock still exists on 8.1-RELEASE.
 
 $ cat /etc/pf.conf
 set block-policy return
 block all
 pass out proto { tcp udp } to port domain
 pass out proto tcp to port { http ftp } user root 
 
 My box deadlock at boot i suppose a minimal DNS traffic start cause pf
 to evaluate rules.
 
 without one of theses rules no deadlock.
 
 $ uname -a
 FreeBSD aldo.philpep.org 8.1-RELEASE FreeBSD 8.1-RELEASE #3: Sat Sep 11
 11:05:55 UTC 2010     root@aldo.philpep.org:/usr/obj/usr/src/sys/ALDO
 amd64
 
 (as you can see i've ipv6 and ipv4 enabled)
 
 
 /boot/kernel/kernel text=0x64d7a6 data=0x916e8+0x23f1c8 syms=[0x8+0xabd68+0x8+0x9c3b7]
 /boot/kernel/zfs.ko size 0x1a3098 at 0xc67000
 loading required module 'opensolaris'
 /boot/kernel/opensolaris.ko size 0x3948 at 0xe0b000
 /
 Hit [Enter] to boot immediately, or any other key for command prompt.
 Booting [/boot/kernel/kernel]...               
 KDB: debugger backends: ddb
 KDB: current backend: ddb
 Copyright (c) 1992-2010 The FreeBSD Project.
 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 	The Regents of the University of California. All rights reserved.
 FreeBSD is a registered trademark of The FreeBSD Foundation.
 FreeBSD 8.1-RELEASE #3: Sat Sep 11 11:05:55 UTC 2010
     root@aldo.philpep.org:/usr/obj/usr/src/sys/ALDO amd64
 WARNING: WITNESS option enabled, expect reduced performance.
 WARNING: DIAGNOSTIC option enabled, expect reduced performance.
 Timecounter "i8254" frequency 1193182 Hz quality 0
 CPU: Intel(R) Celeron(R) CPU        E3300  @ 2.50GHz (2493.11-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x1067a  Family = 6  Model = 17  Stepping = 10
   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
   Features2=0x400e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,XSAVE>
   AMD Features=0x20000800<SYSCALL,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant
 real memory  = 4294967296 (4096 MB)
 avail memory = 3841851392 (3663 MB)
 ACPI APIC Table: <110309 APIC2133>
 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 FreeBSD/SMP: 1 package(s) x 2 core(s)
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
 ioapic0 <Version 2.0> irqs 0-23 on motherboard
 kbd1 at kbdmux0
 cryptosoft0: <software crypto> on motherboard
 acpi0: <110309 RSDT2133> on motherboard
 acpi0: [ITHREAD]
 acpi0: Power Button (fixed)
 acpi0: reservation of 0, a0000 (3) failed
 acpi0: reservation of 100000, cfd00000 (3) failed
 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
 cpu0: <ACPI CPU> on acpi0
 cpu1: <ACPI CPU> on acpi0
 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 pci0: <ACPI PCI bus> on pcib0
 vgapci0: <VGA-compatible display> port 0xdc00-0xdc07 mem 0xfe400000-0xfe7fffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
 agp0: <Intel G41 SVGA controller> on vgapci0
 agp0: detected 262140k stolen memory
 agp0: aperture size is 256M
 pci0: <multimedia, HDA> at device 27.0 (no driver attached)
 pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
 pci2: <ACPI PCI bus> on pcib1
 pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
 pci1: <ACPI PCI bus> on pcib2
 alc0: <Atheros AR8132 PCIe Fast Ethernet> port 0xec00-0xec7f mem 0xfebc0000-0xfebfffff irq 17 at device 0.0 on pci1
 alc0: 15872 Tx FIFO, 15360 Rx FIFO
 alc0: Using 1 MSI message(s).
 alc0: 4GB boundary crossed, switching to 32bit DMA addressing mode.
 miibus0: <MII bus> on alc0
 atphy0: <Atheros F1 10/100/1000 PHY> PHY 0 on miibus0
 atphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 alc0: Ethernet address: 00:25:22:15:70:2b
 alc0: [FILTER]
 uhci0: <Intel 82801G (ICH7) USB controller USB-A> port 0xd400-0xd41f irq 23 at device 29.0 on pci0
 uhci0: [ITHREAD]
 uhci0: LegSup = 0x2f00
 usbus0: <Intel 82801G (ICH7) USB controller USB-A> on uhci0
 uhci1: <Intel 82801G (ICH7) USB controller USB-B> port 0xd480-0xd49f irq 19 at device 29.1 on pci0
 uhci1: [ITHREAD]
 uhci1: LegSup = 0x2f00
 usbus1: <Intel 82801G (ICH7) USB controller USB-B> on uhci1
 uhci2: <Intel 82801G (ICH7) USB controller USB-C> port 0xd800-0xd81f irq 18 at device 29.2 on pci0
 uhci2: [ITHREAD]
 uhci2: LegSup = 0x2f00
 usbus2: <Intel 82801G (ICH7) USB controller USB-C> on uhci2
 uhci3: <Intel 82801G (ICH7) USB controller USB-D> port 0xd880-0xd89f irq 16 at device 29.3 on pci0
 uhci3: [ITHREAD]
 uhci3: LegSup = 0x2f00
 usbus3: <Intel 82801G (ICH7) USB controller USB-D> on uhci3
 ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem 0xfeaf7c00-0xfeaf7fff irq 23 at device 29.7 on pci0
 ehci0: [ITHREAD]
 usbus4: EHCI version 1.0
 usbus4: <Intel 82801GB/R (ICH7) USB 2.0 controller> on ehci0
 pcib3: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 pci3: <ACPI PCI bus> on pcib3
 isab0: <PCI-ISA bridge> at device 31.0 on pci0
 isa0: <ISA bus> on isab0
 atapci0: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0
 ata0: <ATA channel 0> on atapci0
 ata0: [ITHREAD]
 atapci1: <Intel ICH7 SATA300 controller> port 0xd080-0xd087,0xd000-0xd003,0xcc00-0xcc07,0xc880-0xc883,0xc800-0xc80f irq 19 at device 31.2 on pci0
 atapci1: [ITHREAD]
 ata2: <ATA channel 0> on atapci1
 ata2: [ITHREAD]
 ata3: <ATA channel 1> on atapci1
 ata3: [ITHREAD]
 pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 acpi_button0: <Power Button> on acpi0
 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 uart0: [FILTER]
 uart0: console (9600,n,8,1)
 sc0: <System console> at flags 0x100 on isa0
 sc0: VGA <16 virtual consoles, flags=0x100>
 vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
 atkbd0: <AT Keyboard> irq 1 on atkbdc0
 kbd0 at atkbd0
 atkbd0: [GIANT-LOCKED]
 atkbd0: [ITHREAD]
 coretemp0: <CPU On-Die Thermal Sensors> on cpu0
 est0: <Enhanced SpeedStep Frequency Control> on cpu0
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr 6164c2406004a24
 device_attach: est0 attach returned 6
 p4tcc0: <CPU Frequency Thermal Control> on cpu0
 coretemp1: <CPU On-Die Thermal Sensors> on cpu1
 est1: <Enhanced SpeedStep Frequency Control> on cpu1
 est: CPU supports Enhanced Speedstep, but is not recognized.
 est: cpu_vendor GenuineIntel, msr 6164c2406004a24
 device_attach: est1 attach returned 6
 p4tcc1: <CPU Frequency Thermal Control> on cpu1
 ZFS filesystem version 3
 ZFS storage pool version 14
 Timecounters tick every 1.000 msec
 usbus0: 12Mbps Full Speed USB v1.0
 usbus1: 12Mbps Full Speed USB v1.0
 usbus2: 12Mbps Full Speed USB v1.0
 usbus3: 12Mbps Full Speed USB v1.0
 usbus4: 480Mbps High Speed USB v2.0
 ad4: 476940MB <WDC WD5000AAKS-00WWPA0 01.03B01> at ata2-master UDMA100 SATA
 ugen0.1: <Intel> at usbus0
 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
 Expensive timeout(9) function: 0xffffffff8042a350(0xffffff0001900ba0) 0.119308459 s
 ugen1.1: <Intel> at usbus1
 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
 ugen2.1: <Intel> at usbus2
 uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
 ugen3.1: <Intel> at usbus3
 uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
 ugen4.1: <Intel> at usbus4
 uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
 SMP: AP CPU #1 Launched!
 WARNING: WITNESS option enabled, expect reduced performance.
 WARNING: DIAGNOSTIC option enabled, expect reduced performance.
 Root mount waiting for: usbus4 usbus3 usbus2 usbus1 usbus0
 uhub0: 2 ports with 2 removable, self powered
 uhub1: 2 ports with 2 removable, self powered
 uhub2: 2 ports with 2 removable, self powered
 uhub3: 2 ports with 2 removable, self powered
 Root mount waiting for: usbus4
 Root mount waiting for: usbus4
 uhub4: 8 ports with 8 removable, self powered
 Trying to mount root from zfs:tank/root
 Setting hostuuid: 00020003-0004-0005-0006-000700080009.
 Setting hostid: 0x81f4ec68.
 Entropy harvesting: interrupts ethernet point_to_point kickstart.
 Starting file system checks:
 Mounting local file systems:.
 Setting hostname: aldo.philpep.org.
 alc0: link state changed to DOWN
 Starting Network: lo0 alc0.
 lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
 	options=3<RXCSUM,TXCSUM>
 	inet 127.0.0.1 netmask 0xff000000 
 	inet6 ::1 prefixlen 128 
 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
 	nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
 alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
 	ether 00:25:22:15:70:2b
 	inet 192.168.0.20 netmask 0xffffff00 broadcast 192.168.0.255
 	inet6 fe80::225:22ff:fe15:702b%alc0 prefixlen 64 tentative scopeid 0x1 
 	nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
 	media: Ethernet autoselect (none)
 	status: no carrier
 Enabling pfpf enabled
 .
 add net default: gateway 192.168.0.1
 add net ::ffff:0.0.0.0: gateway ::1
 add net ::0.0.0.0: gateway ::1
 net.inet6.ip6.forwarding: 0 -> 0
 alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
 	inet6 fe80::225:22ff:fe15:702b%alc0 prefixlen 64 tentative scopeid 0x1 
 	inet6 2001:470:1f15:7bf:dead:c0de:0:20 prefixlen 96 tentative 
 lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
 	options=3<RXCSUM,TXCSUM>
 	inet6 ::1 prefixlen 128 
 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
 pflog0: flags=0<> metric 0 mtu 33152
 alc0: link state changed to UP
 add net fe80::: gateway ::1
 add net ff02::: gateway ::1
 add net default: gateway 2001:470:1f15:7bf:dead:c0de::1
 IPv4 mapped IPv6 address support=NO
 Starting devd.
 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/zsh
 32-bit compatibility ldconfig path:
 Creating and/or trimming log files.
 Starting syslogd.
 Clearing /tmp (X related).
 Recovering vi editor sessions:lock order reversal:
  1st 0xffffffff80ac42a8 PFil hook read/write mutex (PFil hook read/write mutex) @ /usr/src/sys/net/pfil.c:77
  2nd 0xffffffff80ac5aa8 udp (udp) @ /usr/src/sys/contrib/pf/net/pf.c:3035
 KDB: stack backtrace:
 db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
 _witness_debugger() at _witness_debugger+0x2e
 witness_checkorder() at witness_checkorder+0x81e
 _rw_rlock() at _rw_rlock+0x5f
 pf_socket_lookup() at pf_socket_lookup+0x1c7
 pf_test_udp() at pf_test_udp+0x8ce
 pf_test() at pf_test+0x1125
 pf_check_in() at pf_check_in+0x39
 pfil_run_hooks() at pfil_run_hooks+0xcf
 ip_input() at ip_input+0x2cd
 swi_net() at swi_net+0x151
 intr_event_execute_handlers() at intr_event_execute_handlers+0x66
 ithread_loop() at ithread_loop+0xb2
 fork_exit() at fork_exit+0x12a
 fork_trampoline() at fork_trampoline+0xe
 --- trap 0, rip = 0, rsp = 0xffffff8000041d30, rbp = 0 ---
 panic: _rw_rlock (tcp): wlock already held @ /usr/src/sys/contrib/pf/net/pf.c:3059
 cpuid = 1
 KDB: enter: panic
 [thread pid 856 tid 100103 ]
 Stopped at      kdb_enter+0x3d: movq    $0,0x4fdd00(%rip)
 db> 
 
 
 db> ps
   pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
   856   851    25    25  R+      CPU 1                       sendmail
   851    25    25     0  S+      wait     0xffffff00035ff460 sh
   720     1   720     0  Ss      select   0xffffff0003985b40 syslogd
   582     1   582     0  Ss      select   0xffffff00036d4940 devd
    25     1    25     0  Ss+     wait     0xffffff00034938c0 sh
    24     0     0     0  SL      flowclea 0xffffffff80abf5d0 [flowcleaner]
    23     0     0     0  SL      sdflush  0xffffffff80ad0ef8 [softdepflush]
    22     0     0     0  SL      vlruwt   0xffffff0003065000 [vnlru]
    21     0     0     0  SL      syncer   0xffffffff80abf320 [syncer]
    20     0     0     0  SL      psleep   0xffffffff80abee48 [bufdaemon]
    19     0     0     0  SL      pgzero   0xffffffff80ad298c [pagezero]
    18     0     0     0  SL      psleep   0xffffffff80ad1d28 [vmdaemon]
    17     0     0     0  SL      psleep   0xffffffff80ad1cec [pagedaemon]
    16     0     0     0  SL      ccb_scan 0xffffffff808e0760 [xpt_thrd]
    15     0     0     0  SL      jsw:wait 0xffffffff808fa450 [g_journal switcher]
     9     0     0     0  SL      pftm     0xffffffff801d8d80 [pfpurge]
     8     0     0     0  SL      waiting_ 0xffffffff80ac48e0 [sctp_iterator]
     7     0     0     0  SL      (threaded)                  zfskern
 100098                   D       tx->tx_s 0xffffff0003196a40 [txg_thread_enter]
 100097                   D       tx->tx_q 0xffffff0003196a60 [txg_thread_enter]
 100057                   D       l2arc_fe 0xffffffff80d5ffc0 [l2arc_feed_thread]
 100056                   D       arc_recl 0xffffffff80d50120 [arc_reclaim_thread]
    14     0     0     0  SL      (threaded)                  usb
 [... usb bus ...]
     6     0     0     0  SL      crypto_r 0xffffffff80acfd40 [crypto returns]
     5     0     0     0  SL      crypto_w 0xffffffff80acfd00 [crypto]
    13     0     0     0  SL      -        0xffffffff808fdd44 [yarrow]
     4     0     0     0  SL      -        0xffffffff808fa3c8 [g_down]
     3     0     0     0  SL      -        0xffffffff808fa3c0 [g_up]
     2     0     0     0  SL      -        0xffffffff808fa3b0 [g_event]
    12     0     0     0  WL      (threaded)                  intr
 100053                   I                                   [irq1: atkbd0]
 100052                   I                                   [swi0: uart]
 100051                   I                                   [irq14: ata0]
 100042                   I                                   [irq16: uhci3]
 100037                   I                                   [irq18: uhci2]
 100032                   I                                   [irq19: uhci1+]
 100027                   I                                   [irq23: uhci0 ehci0]
 100025                   I                                   [irq9: acpi0]
 100023                   I                                   [swi2: cambio]
 100019                   I                                   [swi6: task queue]
 100018                   I                                   [swi6: Giant taskq]
 100016                   I                                   [swi5: +]
 100008                   I                                   [swi4: clock]
 100007                   I                                   [swi4: clock]
 100006                   I                                   [swi3: vm]
 100005                   I                                   [swi1: netisr 0]
    11     0     0     0  RL      (threaded)                  idle
 100004                   Run     CPU 0                       [idle: cpu0]
 100003                   CanRun                              [idle: cpu1]
     1     0     1     0  SLs     wait     0xffffff000149d8c0 [init]
    10     0     0     0  SL      audit_wo 0xffffffff80ad0250 [audit]
     0     0     0     0  SLs     (threaded)                  kernel
 100141                   D       -        0xffffff00036d5a00 [zil_clean]
 [... zil_clean ...]
 100096                   D       -        0xffffff00030b8000 [zfs_vn_rele_taskq]
 100095                   D       -        0xffffff00030b8700 [spa_zio]
 [... spa_zio ... ]
 100055                   D       -        0xffffff0001fb4c00 [system_taskq_1]
 100054                   D       -        0xffffff0001fb4c00 [system_taskq_0]
 100026                   D       -        0xffffff0001700600 [alc0 taskq]
 100024                   D       -        0xffffff0001589c80 [kqueue taskq]
 100022                   D       -        0xffffff000158a100 [acpi_task_2]
 100021                   D       -        0xffffff000158a100 [acpi_task_1]
 100020                   D       -        0xffffff000158a100 [acpi_task_0]
 100017                   D       -        0xffffff000152ea00 [thread taskq]
 100012                   D       -        0xffffff000149ad80 [firmware taskq]
 100000                   D       sched    0xffffffff808fa5c0 [swapper]
 
 
 if you're interested by a proccess trace just ask.
State-Changed-From-To: open->patched 
State-Changed-By: glebius 
State-Changed-When: Tue Sep 11 10:56:15 UTC 2012 
State-Changed-Why:  
I believe this is fixed in head. 


Responsible-Changed-From-To: freebsd-pf->glebius 
Responsible-Changed-By: glebius 
Responsible-Changed-When: Tue Sep 11 10:56:15 UTC 2012 
Responsible-Changed-Why:  
I believe this is fixed in head. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=127439 

From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Geoffrey Mainland <mainland@apeiron.net>,
        Philippe Pepiot <phil@philpep.org>
Cc: bug-followup@FreeBSD.org
Subject: kern/127439: [pf] deadlock in pf
Date: Tue, 11 Sep 2012 14:58:05 +0400

   Geoffrey and Philippe,
 
   I've changed PR status to patched, because I expect that the problem
 is no longer present in head branch of FreeBSD.
 
   I'd appreciate if you test that.
 
 -- 
 Totus tuus, Glebius.
State-Changed-From-To: patched->closed 
State-Changed-By: glebius 
State-Changed-When: Wed Jan 22 09:16:40 UTC 2014 
State-Changed-Why:  
Fixed in 10.0-RELEASE. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=127439 
>Unformatted:
