From nobody@FreeBSD.org  Mon Mar 14 16:09:16 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4395E106564A
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 14 Mar 2011 16:09:16 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 17A148FC14
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 14 Mar 2011 16:09:16 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p2EG9Fgc030101
	for <freebsd-gnats-submit@FreeBSD.org>; Mon, 14 Mar 2011 16:09:15 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p2EG9FVP030100;
	Mon, 14 Mar 2011 16:09:15 GMT
	(envelope-from nobody)
Message-Id: <201103141609.p2EG9FVP030100@red.freebsd.org>
Date: Mon, 14 Mar 2011 16:09:15 GMT
From: Andrew Boyer <aboyer@averesystems.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [patch] Panic in in_scrubprefix() during 'ifconfig delete'
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         155555
>Category:       kern
>Synopsis:       [netinet] [patch] Panic in in_scrubprefix() during 'ifconfig delete'
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pluknet
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 14 16:10:15 UTC 2011
>Closed-Date:    Mon Apr 04 09:57:59 UTC 2011
>Last-Modified:  Mon Apr  4 10:00:20 UTC 2011
>Originator:     Andrew Boyer
>Release:        current, 8.2
>Organization:
Avere Systems, Inc.
>Environment:
amd64
>Description:
The system can panic in in_scrubprefix() in sys/netinet/in.c during system shutdown or anytime many addresses are being removed in parallel.

When a route is being reassigned to a new interface address, the ia is used unlocked and unreferenced.  If enough addresses are being torn down simultaneously from the same subnet, the ia can be deleted while the modified route is being set up.

The panic is on line 1184 in the rtinitflags() macro.  This is the backtrace:
panic: general protection fault
cpuid = 6
KDB: stack backtrace:
gdb_trace_self_wrapper() at 0xffffffff803f68ba =
    gdb_trace_self_wrapper+0x2a
panic() at 0xffffffff80459a52 = panic+0x182
trap_fatal() at 0xffffffff80672e0d = trap_fatal+0x2ad
trap() at 0xffffffff80673a80 = trap+0x160
calltrap() at 0xffffffff80658ace = calltrap+0x8
--- trap 0x9, rip = 0xffffffff80518c70,
    rsp = 0xffffffffd119c900,
    rbp = 0xffffffffd119c930 ---
in_ifscrub() at 0xffffffff80518c70 = in_ifscrub+0x100
in_control() at 0xffffffff8051a1a8 = in_control+0x808
ifioctl() at 0xffffffff804f4f0f = ifioctl+0x2af
kern_ioctl() at 0xffffffff80496013 = kern_ioctl+0xa3
ioctl() at 0xffffffff8049627a = ioctl+0xfa
syscall() at 0xffffffff80673460 = syscall+0x2c0
Xfast_syscall() at 0xffffffff80658cdb = Xfast_syscall+0xab
--- syscall (54, FreeBSD ELF64, ioctl),
    rip = 0x800782a8c,
    rsp = 0x7fffffffe618,
    rbp = 0x7fffffffe680 ---
--- curthread 0xffffff000933f370, tid 100301
    pid 6602 "ifconfig"
Stopping all network interfaces

In this case the ifconfig command was 'ifconfig e3c 169.254.6.15/16 delete'.
>How-To-Repeat:
Typing 'reboot' on a system running avahi-autoipd for zeroconf networking on all (8+) interfaces leads to a panic approximately 2-3% of the time.

This is time-consuming to reproduce but the fix is straightforward (see patch below).

It might be possible to reproduce just with killing and restarting avahi-autoipd but I haven't tried it.
>Fix:


Patch attached with submission follows:

Index: sys/netinet/in.c
===================================================================
--- sys/netinet/in.c	(revision 219641)
+++ sys/netinet/in.c	(working copy)
@@ -1176,6 +1176,7 @@
 		if ((ia->ia_flags & IFA_ROUTE) == 0
 		    && (ia->ia_ifp->if_type != IFT_CARP)
 							) {
+			ifa_ref(&ia->ia_ifa);
 			IN_IFADDR_RUNLOCK();
 			rtinit(&(target->ia_ifa), (int)RTM_DELETE,
 			    rtinitflags(target));
@@ -1185,6 +1186,7 @@
 			    rtinitflags(ia) | RTF_UP);
 			if (error == 0)
 				ia->ia_flags |= IFA_ROUTE;
+			ifa_free(&ia->ia_ifa);
 			return (error);
 		}
 	}


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Mon Mar 14 16:15:15 UTC 2011 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155555 
Responsible-Changed-From-To: freebsd-net->pluknet 
Responsible-Changed-By: pluknet 
Responsible-Changed-When: Mon Mar 21 13:00:46 UTC 2011 
Responsible-Changed-Why:  
Take. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155555 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/155555: commit references a PR
Date: Mon, 21 Mar 2011 14:19:53 +0000 (UTC)

 Author: pluknet
 Date: Mon Mar 21 14:19:40 2011
 New Revision: 219828
 URL: http://svn.freebsd.org/changeset/base/219828
 
 Log:
   Reference ifaddr object before unlocking as it can be freed
   from another context at the moment of later access.
   
   PR:		kern/155555
   Submitted by:	Andrew Boyer <aboyer att averesystems.com>
   Approved by:	avg (mentor)
   MFC after:	2 weeks
 
 Modified:
   head/sys/netinet/in.c
 
 Modified: head/sys/netinet/in.c
 ==============================================================================
 --- head/sys/netinet/in.c	Mon Mar 21 14:18:40 2011	(r219827)
 +++ head/sys/netinet/in.c	Mon Mar 21 14:19:40 2011	(r219828)
 @@ -1174,8 +1174,8 @@ in_scrubprefix(struct in_ifaddr *target)
  		 *      doesn't support such action.
  		 */
  		if ((ia->ia_flags & IFA_ROUTE) == 0
 -		    && (ia->ia_ifp->if_type != IFT_CARP)
 -							) {
 +		    && (ia->ia_ifp->if_type != IFT_CARP)) {
 +			ifa_ref(&ia->ia_ifa);
  			IN_IFADDR_RUNLOCK();
  			rtinit(&(target->ia_ifa), (int)RTM_DELETE,
  			    rtinitflags(target));
 @@ -1185,6 +1185,7 @@ in_scrubprefix(struct in_ifaddr *target)
  			    rtinitflags(ia) | RTF_UP);
  			if (error == 0)
  				ia->ia_flags |= IFA_ROUTE;
 +			ifa_free(&ia->ia_ifa);
  			return (error);
  		}
  	}
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: pluknet 
State-Changed-When: Mon Mar 21 14:20:03 UTC 2011 
State-Changed-Why:  
Committed to head with MFC in 2 weeks. Thanks! 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155555 
State-Changed-From-To: patched->closed 
State-Changed-By: pluknet 
State-Changed-When: Mon Apr 4 09:57:20 UTC 2011 
State-Changed-Why:  
Merged to 8-STABLE. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155555 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/155555: commit references a PR
Date: Mon,  4 Apr 2011 09:56:23 +0000 (UTC)

 Author: pluknet
 Date: Mon Apr  4 09:56:10 2011
 New Revision: 220322
 URL: http://svn.freebsd.org/changeset/base/220322
 
 Log:
   MFC r219828:
   
    Reference ifaddr object before unlocking as it can be freed
    from another context at the moment of later access.
   
   PR:		kern/155555
   Submitted by:	Andrew Boyer <aboyer att averesystems.com>
 
 Modified:
   stable/8/sys/netinet/in.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/amd64/include/xen/   (props changed)
   stable/8/sys/cddl/contrib/opensolaris/   (props changed)
   stable/8/sys/contrib/dev/acpica/   (props changed)
   stable/8/sys/contrib/pf/   (props changed)
 
 Modified: stable/8/sys/netinet/in.c
 ==============================================================================
 --- stable/8/sys/netinet/in.c	Mon Apr  4 09:38:05 2011	(r220321)
 +++ stable/8/sys/netinet/in.c	Mon Apr  4 09:56:10 2011	(r220322)
 @@ -1174,8 +1174,8 @@ in_scrubprefix(struct in_ifaddr *target)
  		 *      doesn't support such action.
  		 */
  		if ((ia->ia_flags & IFA_ROUTE) == 0
 -		    && (ia->ia_ifp->if_type != IFT_CARP)
 -							) {
 +		    && (ia->ia_ifp->if_type != IFT_CARP)) {
 +			ifa_ref(&ia->ia_ifa);
  			IN_IFADDR_RUNLOCK();
  			rtinit(&(target->ia_ifa), (int)RTM_DELETE,
  			    rtinitflags(target));
 @@ -1185,6 +1185,7 @@ in_scrubprefix(struct in_ifaddr *target)
  			    rtinitflags(ia) | RTF_UP);
  			if (error == 0)
  				ia->ia_flags |= IFA_ROUTE;
 +			ifa_free(&ia->ia_ifa);
  			return (error);
  		}
  	}
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
>Unformatted:
