From petefrench@ticketswitch.com  Thu Aug 19 13:01:00 2010
Return-Path: <petefrench@ticketswitch.com>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 1444410656C7
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 19 Aug 2010 13:01:00 +0000 (UTC)
	(envelope-from petefrench@ticketswitch.com)
Received: from constantine.ticketswitch.com (constantine.ticketswitch.com [IPv6:2002:57e0:1d4e:1::3])
	by mx1.freebsd.org (Postfix) with ESMTP id CD0778FC27
	for <FreeBSD-gnats-submit@freebsd.org>; Thu, 19 Aug 2010 13:00:59 +0000 (UTC)
Received: from dilbert.rattatosk ([10.64.50.6] helo=dilbert.ticketswitch.com)
	by constantine.ticketswitch.com with esmtps (TLSv1:AES256-SHA:256)
	(Exim 4.69 (FreeBSD))
	(envelope-from <petefrench@ticketswitch.com>)
	id 1Om4jq-000Lqv-No
	for FreeBSD-gnats-submit@freebsd.org; Thu, 19 Aug 2010 14:00:58 +0100
Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.71 (FreeBSD))
	(envelope-from <petefrench@ticketswitch.com>)
	id 1Om4jq-00074C-Mz
	for FreeBSD-gnats-submit@freebsd.org; Thu, 19 Aug 2010 14:00:58 +0100
Message-Id: <E1Om4jq-00074C-Mz@dilbert.ticketswitch.com>
Date: Thu, 19 Aug 2010 14:00:58 +0100
From: Pete French <petefrench@ticketswitch.com>
Reply-To: Pete French <petefrench@ticketswitch.com>
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: ICMP redirect on causes "panic: rtqkill route really not free"
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         149804
>Category:       kern
>Synopsis:       [icmp] [panic] ICMP redirect on causes "panic: rtqkill route really not free"
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    delphij
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Aug 19 13:10:03 UTC 2010
>Closed-Date:    Mon Oct 11 11:27:08 UTC 2010
>Last-Modified:  Mon Oct 11 11:30:05 UTC 2010
>Originator:     Pete French
>Release:        FreeBSD 8.1-STABLE amd64
>Organization:
Ticket Switch Ltd
>Environment:
System: FreeBSD dilbert.rattatosk 8.1-STABLE FreeBSD 8.1-STABLE #1: Tue Aug 17 12:30:13 BST 2010 petefrench@dilbert.rattatosk:/usr/obj/usr/src/sys/GENERIC amd64


>Description:

	Running 8.1 in an environment with multiple routers that issue ICMP redirects can cause the system to panic with the message given in the subject line.

>How-To-Repeat:

	Install an 8.1 machine intp an envir0onment as above - I have not found a reliable way to reporduce it, but in our case it crashes within a few days.


>Fix:

	setting net.inet.icmp.drop_redirect=1 will prevent this. There is also a patch from Xin LI which appears to fix the problem (or at least mitigate it to the point where we havent seen it since it was applied)

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Sat Aug 21 12:33:29 UTC 2010 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=149804 

From: Pete French <pete@twisted.org.uk>
To: bug-followup@FreeBSD.org,
 petefrench@ticketswitch.com
Cc:  
Subject: Re: kern/149804: [icmp] [panic] ICMP redirect on causes "panic: rtqkill route really not free"
Date: Fri, 17 Sep 2010 13:44:14 +0100

 Here is the patch I am currently using which makes the problem go away. =
 As states above, this was sent to me by Xin Li - it's not my own work!
 
 Index: sys/netinet/in_rmx.c
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 --- sys/netinet/in_rmx.c	(revision 211232)
 +++ sys/netinet/in_rmx.c	(working copy)
 @@ -121,12 +121,13 @@ in_matroute(void *v_arg, struct radix_node_head *h
  	struct radix_node *rn =3D rn_match(v_arg, head);
  	struct rtentry *rt =3D (struct rtentry *)rn;
 =20
 -	/*XXX locking? */
 -	if (rt && rt->rt_refcnt =3D=3D 0) {		/* this is first =
 reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &=3D ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire =3D 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/149804: commit references a PR
Date: Mon, 27 Sep 2010 19:27:05 +0000 (UTC)

 Author: delphij
 Date: Mon Sep 27 19:26:56 2010
 New Revision: 213225
 URL: http://svn.freebsd.org/changeset/base/213225
 
 Log:
   Add a bandaid for a long-standing race condition during route entry
   un-expiring.
   
   The previous version of code have no locking when testing rt_refcnt.
   The result of the lack of locking may result in a condition where
   a routing entry have a reference count but at the same time have
   RTPRF_OURS bit set and an expiration timer.  These would eventually
   lead to a panic:
   
   	panic: rtqkill route really not free
   
   When the system have ICMP redirects accepted from local gateway
   in a moderate frequency, for instance.
   
   Commit this workaround for now until we have some better solution.
   
   PR:		kern/149804
   Reviewed by:	bz
   Tested by:	Zhao Xin, Pete French
   MFC after:	2 weeks
 
 Modified:
   head/sys/netinet/in_rmx.c
   head/sys/netinet6/in6_rmx.c
 
 Modified: head/sys/netinet/in_rmx.c
 ==============================================================================
 --- head/sys/netinet/in_rmx.c	Mon Sep 27 19:03:18 2010	(r213224)
 +++ head/sys/netinet/in_rmx.c	Mon Sep 27 19:26:56 2010	(r213225)
 @@ -121,12 +121,13 @@ in_matroute(void *v_arg, struct radix_no
  	struct radix_node *rn = rn_match(v_arg, head);
  	struct rtentry *rt = (struct rtentry *)rn;
  
 -	/*XXX locking? */
 -	if (rt && rt->rt_refcnt == 0) {		/* this is first reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &= ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire = 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 
 Modified: head/sys/netinet6/in6_rmx.c
 ==============================================================================
 --- head/sys/netinet6/in6_rmx.c	Mon Sep 27 19:03:18 2010	(r213224)
 +++ head/sys/netinet6/in6_rmx.c	Mon Sep 27 19:26:56 2010	(r213225)
 @@ -193,11 +193,13 @@ in6_matroute(void *v_arg, struct radix_n
  	struct radix_node *rn = rn_match(v_arg, head);
  	struct rtentry *rt = (struct rtentry *)rn;
  
 -	if (rt && rt->rt_refcnt == 0) { /* this is first reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &= ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire = 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: delphij 
State-Changed-When: Wed Sep 29 05:55:38 UTC 2010 
State-Changed-Why:  
A bandaid have been committed against -HEAD. 


Responsible-Changed-From-To: freebsd-net->delphij 
Responsible-Changed-By: delphij 
Responsible-Changed-When: Wed Sep 29 05:55:38 UTC 2010 
Responsible-Changed-Why:  
Take. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=149804 
State-Changed-From-To: patched->closed 
State-Changed-By: delphij 
State-Changed-When: Mon Oct 11 11:25:53 UTC 2010 
State-Changed-Why:  
Workaround have been MFC'ed to stable/{7,8}. 

Note that this problem MAY apply to earlier FreeBSD releases which are now 
EoL'ed, since I have not local environment for testing, I'd prefer leave 
these branches as-is for now. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=149804 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/149804: commit references a PR
Date: Mon, 11 Oct 2010 11:25:45 +0000 (UTC)

 Author: delphij
 Date: Mon Oct 11 11:25:37 2010
 New Revision: 213687
 URL: http://svn.freebsd.org/changeset/base/213687
 
 Log:
   MFC r213225:
   
   Add a bandaid for a long-standing race condition during route entry
   un-expiring.
   
   The previous version of code have no locking when testing rt_refcnt.
   The result of the lack of locking may result in a condition where
   a routing entry have a reference count but at the same time have
   RTPRF_OURS bit set and an expiration timer.  These would eventually
   lead to a panic:
   
   	panic: rtqkill route really not free
   
   When the system have ICMP redirects accepted from local gateway
   in a moderate frequency, for instance.
   
   Commit this workaround for now until we have some better solution.
   
   PR:		kern/149804
   Reviewed by:	bz
   Tested by:	Zhao Xin, Pete French
 
 Modified:
   stable/8/sys/netinet/in_rmx.c
   stable/8/sys/netinet6/in6_rmx.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
   stable/8/sys/dev/xen/xenpci/   (props changed)
 
 Changes in other areas also in this revision:
 Modified:
   stable/7/sys/netinet/in_rmx.c
   stable/7/sys/netinet6/in6_rmx.c
 Directory Properties:
   stable/7/sys/   (props changed)
   stable/7/sys/cddl/contrib/opensolaris/   (props changed)
   stable/7/sys/contrib/dev/acpica/   (props changed)
   stable/7/sys/contrib/pf/   (props changed)
 
 Modified: stable/8/sys/netinet/in_rmx.c
 ==============================================================================
 --- stable/8/sys/netinet/in_rmx.c	Mon Oct 11 09:42:30 2010	(r213686)
 +++ stable/8/sys/netinet/in_rmx.c	Mon Oct 11 11:25:37 2010	(r213687)
 @@ -121,12 +121,13 @@ in_matroute(void *v_arg, struct radix_no
  	struct radix_node *rn = rn_match(v_arg, head);
  	struct rtentry *rt = (struct rtentry *)rn;
  
 -	/*XXX locking? */
 -	if (rt && rt->rt_refcnt == 0) {		/* this is first reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &= ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire = 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 
 Modified: stable/8/sys/netinet6/in6_rmx.c
 ==============================================================================
 --- stable/8/sys/netinet6/in6_rmx.c	Mon Oct 11 09:42:30 2010	(r213686)
 +++ stable/8/sys/netinet6/in6_rmx.c	Mon Oct 11 11:25:37 2010	(r213687)
 @@ -193,11 +193,13 @@ in6_matroute(void *v_arg, struct radix_n
  	struct radix_node *rn = rn_match(v_arg, head);
  	struct rtentry *rt = (struct rtentry *)rn;
  
 -	if (rt && rt->rt_refcnt == 0) { /* this is first reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &= ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire = 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/149804: commit references a PR
Date: Mon, 11 Oct 2010 11:25:46 +0000 (UTC)

 Author: delphij
 Date: Mon Oct 11 11:25:37 2010
 New Revision: 213687
 URL: http://svn.freebsd.org/changeset/base/213687
 
 Log:
   MFC r213225:
   
   Add a bandaid for a long-standing race condition during route entry
   un-expiring.
   
   The previous version of code have no locking when testing rt_refcnt.
   The result of the lack of locking may result in a condition where
   a routing entry have a reference count but at the same time have
   RTPRF_OURS bit set and an expiration timer.  These would eventually
   lead to a panic:
   
   	panic: rtqkill route really not free
   
   When the system have ICMP redirects accepted from local gateway
   in a moderate frequency, for instance.
   
   Commit this workaround for now until we have some better solution.
   
   PR:		kern/149804
   Reviewed by:	bz
   Tested by:	Zhao Xin, Pete French
 
 Modified:
   stable/7/sys/netinet/in_rmx.c
   stable/7/sys/netinet6/in6_rmx.c
 Directory Properties:
   stable/7/sys/   (props changed)
   stable/7/sys/cddl/contrib/opensolaris/   (props changed)
   stable/7/sys/contrib/dev/acpica/   (props changed)
   stable/7/sys/contrib/pf/   (props changed)
 
 Changes in other areas also in this revision:
 Modified:
   stable/8/sys/netinet/in_rmx.c
   stable/8/sys/netinet6/in6_rmx.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
   stable/8/sys/dev/xen/xenpci/   (props changed)
 
 Modified: stable/7/sys/netinet/in_rmx.c
 ==============================================================================
 --- stable/7/sys/netinet/in_rmx.c	Mon Oct 11 09:42:30 2010	(r213686)
 +++ stable/7/sys/netinet/in_rmx.c	Mon Oct 11 11:25:37 2010	(r213687)
 @@ -140,12 +140,13 @@ in_matroute(void *v_arg, struct radix_no
  	struct radix_node *rn = rn_match(v_arg, head);
  	struct rtentry *rt = (struct rtentry *)rn;
  
 -	/*XXX locking? */
 -	if (rt && rt->rt_refcnt == 0) {		/* this is first reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &= ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire = 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 
 Modified: stable/7/sys/netinet6/in6_rmx.c
 ==============================================================================
 --- stable/7/sys/netinet6/in6_rmx.c	Mon Oct 11 09:42:30 2010	(r213686)
 +++ stable/7/sys/netinet6/in6_rmx.c	Mon Oct 11 11:25:37 2010	(r213687)
 @@ -207,11 +207,13 @@ in6_matroute(void *v_arg, struct radix_n
  	struct radix_node *rn = rn_match(v_arg, head);
  	struct rtentry *rt = (struct rtentry *)rn;
  
 -	if (rt && rt->rt_refcnt == 0) { /* this is first reference */
 +	if (rt) {
 +		RT_LOCK(rt);
  		if (rt->rt_flags & RTPRF_OURS) {
  			rt->rt_flags &= ~RTPRF_OURS;
  			rt->rt_rmx.rmx_expire = 0;
  		}
 +		RT_UNLOCK(rt);
  	}
  	return rn;
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
