From nobody@FreeBSD.org  Wed Dec 26 10:22:13 2007
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 8FAD416A418
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 26 Dec 2007 10:22:13 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 8B2F213C45D
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 26 Dec 2007 10:22:13 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.2/8.14.2) with ESMTP id lBQALcSw039468
	for <freebsd-gnats-submit@FreeBSD.org>; Wed, 26 Dec 2007 10:21:38 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.2/8.14.1/Submit) id lBQALcUw039467;
	Wed, 26 Dec 2007 10:21:38 GMT
	(envelope-from nobody)
Message-Id: <200712261021.lBQALcUw039467@www.freebsd.org>
Date: Wed, 26 Dec 2007 10:21:38 GMT
From: Manuel Kasper <mk@neon1.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [patch] enc(4) and dummynet together produce kernel panics
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         119036
>Category:       kern
>Synopsis:       [netipsec] [patch] enc(4) and dummynet together produce kernel panics
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    thompsa
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Dec 26 10:30:01 UTC 2007
>Closed-Date:    Mon Jan 07 20:32:25 UTC 2008
>Last-Modified:  Mon Jan 07 20:32:25 UTC 2008
>Originator:     Manuel Kasper
>Release:        6.2-RELEASE-p9
>Organization:
>Environment:
FreeBSD fb62.neon1.net 6.2-RELEASE-p9 FreeBSD 6.2-RELEASE-p9 #0: Sun Dec 23 15:09:45 CET 2007     root@fb62.neon1.net:/usr/src/sys/i386/compile/GENERIC_IPSEC_ENC  i386

>Description:
When enc(4) is used in conjunction with ipfw+dummynet, certain configurations may lead to a kernel panic like the following:

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x4
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0xc05786aa
stack pointer           = 0x28:0xc8798a9c
frame pointer           = 0x28:0xc8798aac
code segment            = base 0x0, limit 0xfffff, type 0x1b
                       = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 11 (swi1: net)
trap number             = 12
panic: page fault

This happens if all of the following conditions are true:

- FAST_IPSEC is used (which is a prerequisite for enc(4) anyway)

- at least one IPsec SPD and SAD entry is present

- ipfw and dummynet are loaded

- there is an ipfw rule that matches incoming decapsulated IPsec packets (not ESP or IPIP, but the actual payload) on the interface where ESP packets come in

- interface enc0 is up

Then, as soon as one ESP packet is received, the kernel panics.

Kernel debugging showed that the problem lies in the three different places where ipsec_filter() (from sys/net/if_enc.c) is called: in sys/netipsec/{ipsec_input.c,ipsec_output.c,xform_ipip.c}. When ipsec_filter() passes a packet (via pfil hooks) to ipfw, and it gets consumed by dummynet, ipsec_filter() returns 0 and the mbuf pointer that was passed in is set to NULL. However, the ipsec_filter() callers do not check for this and continue processing with the NULL mbuf.

The fix is to add appropriate checks to stop processing if the mbuf pointer was set to NULL.

Furthermore, it seems that ipfw doesn't match inbound packets on enc0, but on the interface where the corresponding ESP packet came in instead; this is not surprising though, as ipfw_check_in() in sys/netinet/ip_fw_pfil.c ignores its "ifp" argument, and if_enc doesn't replace the ifp in the mbuf with the one for enc0. It works for outbound packets on enc0 as ipfw_check_out() honors the ifp argument.
>How-To-Repeat:
On a FreeBSD 6.2 machine, compile a kernel with

options FAST_IPSEC
device enc
device crypto

Set up an IPsec tunnel to a remote system (manually with setkey or using ipsec-tools/racoon); assume that the local endpoint is on interface em0. Load ipfw and dummynet, and then add rules as follows:

ipfw pipe 100 config bw 10Mbps
ipfw add 100 pipe 100 ip from any to any in via em0
ipfw add 60000 pass ip from any to any

Configure enc0 up:

ifconfig enc0 up

Send a single packet through the IPsec tunnel from the remote system (or just ping the remote system from the machine under test), and you'll get a kernel panic immediately.
>Fix:
See the attached patch for the kernel panic issue (but not the general inconsistency of ipfw not matching inbound packets on enc0).

Patch attached with submission follows:

--- sys/netipsec/ipsec_input.c.orig	Tue Jul 25 01:20:59 2006
+++ sys/netipsec/ipsec_input.c	Sun Dec 23 16:20:07 2007
@@ -450,9 +450,12 @@
 	 */
 	ipsec_bpf(m, sav, AF_INET);
 
-	if (prot != IPPROTO_IPIP)
+	if (prot != IPPROTO_IPIP) {
 		if ((error = ipsec_filter(&m, 1)) != 0)
 			return (error);
+		if (m == NULL)		/* consumed by filter */
+			return 0;
+	}
 #endif
 
 	/*
--- sys/netipsec/ipsec_output.c.orig	Tue Jul 25 01:20:59 2006
+++ sys/netipsec/ipsec_output.c	Sun Dec 23 16:20:13 2007
@@ -364,6 +364,8 @@
 	/* pass the mbuf to enc0 for packet filtering */
 	if ((error = ipsec_filter(&m, 2)) != 0)
 		goto bad;
+	if (m == NULL)		/* consumed by filter */
+		goto bad;
 #endif
 
 	if (!tunalready) {
--- sys/netipsec/xform_ipip.c.orig	Sat Dec  9 14:05:49 2006
+++ sys/netipsec/xform_ipip.c	Sun Dec 23 16:20:22 2007
@@ -350,6 +350,8 @@
 	/* pass the mbuf to enc0 for packet filtering */
 	if (ipsec_filter(&m, 1) != 0)
 		return;
+	if (m == NULL)		/* consumed by filter */
+		return;
 #endif
 
 	/*


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Dec 26 12:04:44 UTC 2007 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=119036 
Responsible-Changed-From-To: freebsd-net->thompsa 
Responsible-Changed-By: thompsa 
Responsible-Changed-When: Wed Dec 26 17:02:25 UTC 2007 
Responsible-Changed-Why:  
I'll grab this one. I was forwarded the local patch from the 
m0n0wall repo a few days ago and have fixed it in a slightly 
differenet way, see r1.8 src/sys/net/if_enc.c 

Thanks for finding/fixing it and the PR. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=119036 
State-Changed-From-To: open->closed 
State-Changed-By: thompsa 
State-Changed-When: Mon Jan 7 20:31:52 UTC 2008 
State-Changed-Why:  
Patch has been merged to all branches. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=119036 
>Unformatted:
