From nobody@FreeBSD.org  Thu Mar  8 22:52:14 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 37133106566C
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  8 Mar 2012 22:52:14 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 0C7258FC0C
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  8 Mar 2012 22:52:14 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q28MqDYw073866
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 8 Mar 2012 22:52:13 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id q28MqDlN073865;
	Thu, 8 Mar 2012 22:52:13 GMT
	(envelope-from nobody)
Message-Id: <201203082252.q28MqDlN073865@red.freebsd.org>
Date: Thu, 8 Mar 2012 22:52:13 GMT
From: Eric van Gyzen <eric_van_gyzen@dell.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: [panic] [patch] in_lltable_prefix_free() races with lla_lookup() and arptimer()
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         165863
>Category:       kern
>Synopsis:       [panic] [netinet] [patch] in_lltable_prefix_free() races with lla_lookup() and arptimer()
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    glebius
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Mar 08 23:00:27 UTC 2012
>Closed-Date:    Mon Sep 10 12:31:28 UTC 2012
>Last-Modified:  Wed Sep 11 07:40:00 UTC 2013
>Originator:     Eric van Gyzen
>Release:        8.2-RELEASE
>Organization:
Dell Compellent
>Environment:
FreeBSD 8.2-RELEASE amd64
>Description:
in_lltable_prefix_free() does not acquire the if_afdata_lock, so it can free the llentry while another thread is in lla_lookup(), such as during packet processing.

While working on a fix, it quickly became apparent that in_lltable_prefix_free() does not safely drain the callout, so it races with arptimer().  If arptimer() has already tested callout_active() before in_lltable_prefix_free() calls callout_drain(), arptimer() will have freed the llentry before callout_drain() returns.

I can reliably reproduce the first problem (lla_lookup) on 8.1-RELEASE and 8.2-RELEASE.  I cannot reproduce it on 9.0-RELEASE or head, but since the relevant code has not changed, I suspect it still exists.

I have not earnestly tried to reproduce the second problem (arptimer).

The same problem exists in the IPv6 code, but that code is never called.
>How-To-Repeat:
In a second process, repeatedly and rapidly call SIOCSIFADDR and SIOCDIFADDR on the /only/ interface address by which that neighbor is reached.  This drives in_lltable_prefix_free().

In one process, flood ping an IPv4 neighbor.  This drives lla_lookup()
via the processing of ARP replies, packets queued on la_hold, and
echo request packets.

This reliably panics in 10-20 seconds.
>Fix:
With the attached patch, my systems survive the test for over 30 minutes.

Patch attached with submission follows:

--- sys/netinet/in.c.orig	2012-03-08 12:57:29.000000000 -0600
+++ sys/netinet/in.c	2012-03-08 12:14:26.000000000 -0600
@@ -1351,21 +1351,39 @@
 	struct llentry *lle, *next;
 	register int i;
 
+restart:
+	IF_AFDATA_WLOCK(llt->llt_ifp);
 	for (i=0; i < LLTBL_HASHTBL_SIZE; i++) {
 		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
 
-			if (IN_ARE_MASKED_ADDR_EQUAL((struct sockaddr_in *)L3_ADDR(lle), 
+			if (IN_ARE_MASKED_ADDR_EQUAL(satosin(L3_ADDR(lle)),
 						     pfx, msk)) {
-				int canceled;
+				int canceled = 0;
 
-				canceled = callout_drain(&lle->la_timer);
 				LLE_WLOCK(lle);
-				if (canceled)
+				if (!callout_active(&lle->la_timer) ||
+				    (canceled = callout_stop(&lle->la_timer))) {
+					if (canceled)
+						LLE_REMREF(lle);
+					llentry_free(lle);
+				} else {
+					/*
+					 * The callout is in progress.
+					 * Since we deactivated it, it won't
+					 * do anything.  Drop its reference.
+					 * Drop our locks to drain it, which
+					 * requires restarting this function.
+					 */
 					LLE_REMREF(lle);
-				llentry_free(lle);
+					LLE_WUNLOCK(lle);
+					IF_AFDATA_WUNLOCK(llt->llt_ifp);
+					(void) callout_drain(&lle->la_timer);
+					goto restart;
+				}
 			}
 		}
 	}
+	IF_AFDATA_WUNLOCK(llt->llt_ifp);
 }
 
 


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Thu Mar 8 23:05:11 UTC 2012 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165863 

From: Eric van Gyzen <eric@vangyzen.net>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/165863: [panic] [netinet] [patch] in_lltable_prefix_free()
 races with lla_lookup() and arptimer()
Date: Thu, 08 Mar 2012 20:23:14 -0600

 I just realized that the first two paragraphs in the How-To-Repeat 
 section are interchanged.  I apologize for the confusion.

From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Eric van Gyzen <eric_van_gyzen@dell.com>,
        Eric van Gyzen <eric@vangyzen.net>, emaste@FreeBSD.org
Cc: bug-followup@FreeBSD.org
Subject: kern/165863
Date: Fri, 9 Mar 2012 13:20:56 +0400

 --BXVAT5kNtrzKuDFl
 Content-Type: text/plain; charset=koi8-r
 Content-Disposition: inline
 
   Hello, Eric and Ed.
 
   Can you look at this patch? I decided to utilize newer callout API,
 that allows to delegate lock retrieval to the callout subsystem, and
 this makes things simplier. Hope that should work.
 
   Patch is against head.
 
   Eric, can you please send me your test programs, that you use to
 reproduce the bug?
 
 -- 
 Totus tuus, Glebius.
 
 --BXVAT5kNtrzKuDFl
 Content-Type: text/x-diff; charset=koi8-r
 Content-Disposition: attachment; filename="kern165863.diff"
 
 Index: in.c
 ===================================================================
 --- in.c	(revision 232689)
 +++ in.c	(working copy)
 @@ -1279,11 +1279,12 @@
  {
  	struct in_llentry *lle;
  
 -	lle = malloc(sizeof(struct in_llentry), M_LLTABLE, M_DONTWAIT | M_ZERO);
 +	lle = malloc(sizeof(struct in_llentry), M_LLTABLE, M_NOWAIT | M_ZERO);
  	if (lle == NULL)		/* NB: caller generates msg */
  		return NULL;
  
 -	callout_init(&lle->base.la_timer, CALLOUT_MPSAFE);
 +	LLE_LOCK_INIT(&lle->base);
 +	callout_init_rw(&lle->base.la_timer, &lle->base.lle_lock, 0);
  	/*
  	 * For IPv4 this will trigger "arpresolve" to generate
  	 * an ARP request.
 @@ -1292,46 +1293,44 @@
  	lle->l3_addr4 = *(const struct sockaddr_in *)l3addr;
  	lle->base.lle_refcnt = 1;
  	lle->base.lle_free = in_lltable_free;
 -	LLE_LOCK_INIT(&lle->base);
 -	return &lle->base;
 +
 +	return (&lle->base);
  }
  
  #define IN_ARE_MASKED_ADDR_EQUAL(d, a, m)	(			\
  	    (((ntohl((d)->sin_addr.s_addr) ^ (a)->sin_addr.s_addr) & (m)->sin_addr.s_addr)) == 0 )
  
  static void
 -in_lltable_prefix_free(struct lltable *llt, 
 -		       const struct sockaddr *prefix,
 -		       const struct sockaddr *mask,
 -		       u_int flags)
 +in_lltable_prefix_free(struct lltable *llt, const struct sockaddr *prefix,
 +    const struct sockaddr *mask, u_int flags)
  {
  	const struct sockaddr_in *pfx = (const struct sockaddr_in *)prefix;
  	const struct sockaddr_in *msk = (const struct sockaddr_in *)mask;
  	struct llentry *lle, *next;
 -	register int i;
 +	int i;
  	size_t pkts_dropped;
  
 -	for (i=0; i < LLTBL_HASHTBL_SIZE; i++) {
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
 +	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  
  		        /* 
  			 * (flags & LLE_STATIC) means deleting all entries
 -			 * including static ARP entries
 +			 * including static ARP entries.
  			 */
 -			if (IN_ARE_MASKED_ADDR_EQUAL((struct sockaddr_in *)L3_ADDR(lle), 
 -						     pfx, msk) &&
 -			    ((flags & LLE_STATIC) || !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 +			if (IN_ARE_MASKED_ADDR_EQUAL(satosin(L3_ADDR(lle)), 
 +			    pfx, msk) && ((flags & LLE_STATIC) ||
 +			    !(lle->la_flags & LLE_STATIC))) {
  
 -				canceled = callout_drain(&lle->la_timer);
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				pkts_dropped = llentry_free(lle);
  				ARPSTAT_ADD(dropped, pkts_dropped);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  
 Index: if_ether.c
 ===================================================================
 --- if_ether.c	(revision 232689)
 +++ if_ether.c	(working copy)
 @@ -163,38 +163,23 @@
  static void
  arptimer(void *arg)
  {
 +	struct llentry *lle = (struct llentry *)arg;
  	struct ifnet *ifp;
 -	struct llentry   *lle;
 -	int pkts_dropped;
 +	size_t pkts_dropped;
  
 -	KASSERT(arg != NULL, ("%s: arg NULL", __func__));
 -	lle = (struct llentry *)arg;
 +	LLE_WLOCK_ASSERT(lle);
 +
 +	if (lle->la_flags & LLE_STATIC)
 +		return;
 +
  	ifp = lle->lle_tbl->llt_ifp;
  	CURVNET_SET(ifp->if_vnet);
 -	IF_AFDATA_LOCK(ifp);
 -	LLE_WLOCK(lle);
 -	if (lle->la_flags & LLE_STATIC)
 -		LLE_WUNLOCK(lle);
 -	else {
 -		if (!callout_pending(&lle->la_timer) &&
 -		    callout_active(&lle->la_timer)) {
 -			callout_stop(&lle->la_timer);
 -			LLE_REMREF(lle);
 -			pkts_dropped = llentry_free(lle);
 -			ARPSTAT_ADD(dropped, pkts_dropped);
 -			ARPSTAT_INC(timeouts);
 -		} else {
 -#ifdef DIAGNOSTIC
 -			struct sockaddr *l3addr = L3_ADDR(lle);
 -			log(LOG_INFO, 
 -			    "arptimer issue: %p, IPv4 address: \"%s\"\n", lle,
 -			    inet_ntoa(
 -			        ((const struct sockaddr_in *)l3addr)->sin_addr));
 -#endif
 -			LLE_WUNLOCK(lle);
 -		}
 -	}
 -	IF_AFDATA_UNLOCK(ifp);
 +
 +	LLE_REMREF(lle);
 +	pkts_dropped = llentry_free(lle);
 +	ARPSTAT_ADD(dropped, pkts_dropped);
 +	ARPSTAT_INC(timeouts);
 +
  	CURVNET_RESTORE();
  }
  
 
 --BXVAT5kNtrzKuDFl--

From: Eric van Gyzen <eric@vangyzen.net>
To: Gleb Smirnoff <glebius@FreeBSD.org>
Cc: Eric van Gyzen <eric_van_gyzen@dell.com>, <emaste@FreeBSD.org>,
	<bug-followup@FreeBSD.org>, <rysto32@gmail.com>
Subject: Re: kern/165863
Date: Fri, 9 Mar 2012 09:26:51 -0600

 --------------000307010007030804050405
 Content-Type: text/plain; charset="KOI8-R"; format=flowed
 Content-Transfer-Encoding: 7bit
 
 On 03/09/12 03:20, Gleb Smirnoff wrote:
 >    Hello, Eric and Ed.
 >
 >    Can you look at this patch? I decided to utilize newer callout API,
 > that allows to delegate lock retrieval to the callout subsystem, and
 > this makes things simplier. Hope that should work.
 >
 >    Patch is against head.
 
 Doesn't arptimer() still need to acquire the if_afdata_lock in order to 
 free the entry in the normal case (when the llentry is still in the hash 
 bucket list)?
 
 With this patch, in_lltable_prefix_free() no longer guarantees that all 
 the relevant llentries will be freed when it returns.  I don't see any 
 immediate breakage, but it's a notable change in behavior.
 
 >    Eric, can you please send me your test programs, that you use to
 > reproduce the bug?
 
 Attached is a C program to add and remove the interface address.  To 
 drive traffic, I just used "ping -f".
 
 Thanks for your help.
 
 Eric
 
 --------------000307010007030804050405
 Content-Type: text/plain; name="ifconf.set.c"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment; filename="ifconf.set.c"
 
 /*-
  * Copyright (c) 2012 Dell, Inc.
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
  * are met:
  * 1. Redistributions of source code must retain the above copyright
  *    notice, this list of conditions and the following disclaimer.
  * 2. Redistributions in binary form must reproduce the above copyright
  *    notice, this list of conditions and the following disclaimer in the
  *    documentation and/or other materials provided with the distribution.
  *
  * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
  * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
  * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
  * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
  * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
  * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
  * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  */
 
 #include <sys/cdefs.h>
 #include <sys/types.h>
 #include <sys/socket.h>
 #include <sys/ioctl.h>
 
 #include <net/if.h>
 
 #include <netinet/in.h>
 
 #include <arpa/inet.h>
 
 #include <err.h>
 #include <errno.h>
 #include <signal.h>
 #include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <strings.h>
 
 int sockfd = -1;
 struct ifreq ifr;
 struct sockaddr_in * const sin4 = (struct sockaddr_in *) &ifr.ifr_addr;
 const char *ifname;
 struct in_addr addr, mask, brdaddr;
 
 unsigned int iterations = 0;
 bool sigint_received = false;
 
 static void
 report(void)
 {
     printf("%u iterations\n", iterations);
 }
 
 static void __dead2
 usage()
 {
     errx(1, "usage: ifconf <ifname> <addr> <mask> <brdaddr> [1]");
 }
 
 static void
 ifconf(void)
 {
     bzero(&ifr, sizeof ifr);
     strlcpy(ifr.ifr_name, ifname, sizeof ifr.ifr_name);
 
     sin4->sin_family = AF_INET;
     sin4->sin_len = sizeof (*sin4);
     sin4->sin_addr = addr;
     if (ioctl(sockfd, SIOCDIFADDR, &ifr)) {
         if (iterations == 0 && errno == EADDRNOTAVAIL) {
             // OK, it wasn't there when we started.
         } else {
             err(1,  "SIOCDIFADDR");
         }
     }
 
     if (ioctl(sockfd, SIOCSIFADDR, &ifr)) {
         err(1,  "SIOCSIFADDR");
     }
 
 #if 0
     sin4->sin_addr = mask;
     if (ioctl(sockfd, SIOCSIFNETMASK, &ifr)) {
         err(1,  "SIOCSIFNETMASK");
     }
 
     sin4->sin_addr = brdaddr;
     if (ioctl(sockfd, SIOCSIFBRDADDR, &ifr)) {
         err(1,  "SIOCSIFBRDADDR");
     }
 #endif
 }
 
 static void
 sigint_handler(int signo __unused)
 {
     sigint_received = true;
 }
 
 int
 main(int argc, const char *argv[])
 {
     if (argc != 5 && argc != 6) {
         usage();
     }
 
     ifname = argv[1];
     if (inet_pton(AF_INET, argv[2], &addr) != 1) {
         err(1, "inet_pton(%s)", argv[2]);
     }
     if (inet_pton(AF_INET, argv[3], &mask) != 1) {
         err(1, "inet_pton(%s)", argv[3]);
     }
     if (inet_pton(AF_INET, argv[4], &brdaddr) != 1) {
         err(1, "inet_pton(%s)", argv[4]);
     }
 
     sockfd = socket(AF_INET, SOCK_DGRAM, 0);
     if (sockfd < 0) {
         err(1, "socket");
     }
 
     if (argc == 6 && atoi(argv[5]) == 1) {
         ifconf();
         return (0);
     }
 
     atexit(report);
     signal(SIGINT, sigint_handler);
 
     while (!sigint_received) {
         ifconf();
         iterations++;
     }
     putchar('\n');
 
     return (0);
 }
 
 --------------000307010007030804050405--

From: Ryan Stone <rysto32@gmail.com>
To: Eric van Gyzen <eric@vangyzen.net>
Cc: Gleb Smirnoff <glebius@freebsd.org>, Eric van Gyzen <eric_van_gyzen@dell.com>, emaste@freebsd.org, 
	bug-followup@freebsd.org
Subject: Re: kern/165863
Date: Tue, 27 Mar 2012 17:24:37 -0400

 2012/3/9 Eric van Gyzen <eric@vangyzen.net>:
 > On 03/09/12 03:20, Gleb Smirnoff wrote:
 >>
 >> =A0 Hello, Eric and Ed.
 >>
 >> =A0 Can you look at this patch? I decided to utilize newer callout API,
 >> that allows to delegate lock retrieval to the callout subsystem, and
 >> this makes things simplier. Hope that should work.
 >>
 >> =A0 Patch is against head.
 >
 >
 > Doesn't arptimer() still need to acquire the if_afdata_lock in order to f=
 ree
 > the entry in the normal case (when the llentry is still in the hash bucke=
 t
 > list)?
 
 Oops, on reviewing the code I believe that this is correct.  My test
 case wouldn't have had arp entries timing out so I wouldn't have hit
 this case.  I'll try to come up with a test case for this.
 
 Unfortunately it's not as quite as simple just acquiring
 if_afdata_lock because of lock ordering problems.  I think that we'll
 have to revert the usage of callout_init_rw so that arptimer can
 acquire the if_afdata_lock before acquiring the lock LLE_LOCK.

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/165863: commit references a PR
Date: Thu,  2 Aug 2012 13:58:02 +0000 (UTC)

 Author: glebius
 Date: Thu Aug  2 13:57:49 2012
 New Revision: 238990
 URL: http://svn.freebsd.org/changeset/base/238990
 
 Log:
   Fix races between in_lltable_prefix_free(), lla_lookup(),
   llentry_free() and arptimer():
   
   o Use callout_init_rw() for lle timeout, this allows us safely
     disestablish them.
     - This allows us to simplify the arptimer() and make it
       race safe.
   o Consistently use ifp->if_afdata_lock to lock access to
     linked lists in the lle hashes.
   o Introduce new lle flag LLE_LINKED, which marks an entry that
     is attached to the hash.
     - Use LLE_LINKED to avoid double unlinking via consequent
       calls to llentry_free().
     - Mark lle with LLE_DELETED via |= operation istead of =,
       so that other flags won't be lost.
   o Make LLE_ADDREF(), LLE_REMREF() and LLE_FREE_LOCKED() more
     consistent and provide more informative KASSERTs.
   
   The patch is a collaborative work of all submitters and myself.
   
   PR:		kern/165863
   Submitted by:	Andrey Zonov <andrey zonov.org>
   Submitted by:	Ryan Stone <rysto32 gmail.com>
   Submitted by:	Eric van Gyzen <eric_van_gyzen dell.com>
 
 Modified:
   head/sys/net/if_llatbl.c
   head/sys/net/if_llatbl.h
   head/sys/net/if_var.h
   head/sys/netinet/if_ether.c
   head/sys/netinet/in.c
   head/sys/netinet6/in6.c
 
 Modified: head/sys/net/if_llatbl.c
 ==============================================================================
 --- head/sys/net/if_llatbl.c	Thu Aug  2 13:20:44 2012	(r238989)
 +++ head/sys/net/if_llatbl.c	Thu Aug  2 13:57:49 2012	(r238990)
 @@ -106,10 +106,19 @@ llentry_free(struct llentry *lle)
  	size_t pkts_dropped;
  	struct mbuf *next;
  
 -	pkts_dropped = 0;
 +	IF_AFDATA_WLOCK_ASSERT(lle->lle_tbl->llt_ifp);
  	LLE_WLOCK_ASSERT(lle);
 +
 +	/* XXX: guard against race with other llentry_free(). */
 +	if (!(lle->la_flags & LLE_LINKED)) {
 +		LLE_FREE_LOCKED(lle);
 +		return (0);
 +	}
 +
  	LIST_REMOVE(lle, lle_next);
 +	lle->la_flags &= ~(LLE_VALID | LLE_LINKED);
  
 +	pkts_dropped = 0;
  	while ((lle->la_numheld > 0) && (lle->la_hold != NULL)) {
  		next = lle->la_hold->m_nextpkt;
  		m_freem(lle->la_hold);
 @@ -122,7 +131,6 @@ llentry_free(struct llentry *lle)
  		("%s: la_numheld %d > 0, pkts_droped %zd", __func__,
  		 lle->la_numheld, pkts_dropped));
  
 -	lle->la_flags &= ~LLE_VALID;
  	LLE_FREE_LOCKED(lle);
  
  	return (pkts_dropped);
 @@ -173,17 +181,16 @@ lltable_free(struct lltable *llt)
  	SLIST_REMOVE(&V_lltables, llt, lltable, llt_link);
  	LLTABLE_WUNLOCK();
  
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
  	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
 -			int canceled;
 -
 -			canceled = callout_drain(&lle->la_timer);
  			LLE_WLOCK(lle);
 -			if (canceled)
 +			if (callout_stop(&lle->la_timer))
  				LLE_REMREF(lle);
  			llentry_free(lle);
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  
  	free(llt, M_LLTABLE);
  }
 
 Modified: head/sys/net/if_llatbl.h
 ==============================================================================
 --- head/sys/net/if_llatbl.h	Thu Aug  2 13:20:44 2012	(r238989)
 +++ head/sys/net/if_llatbl.h	Thu Aug  2 13:57:49 2012	(r238990)
 @@ -103,26 +103,28 @@ struct llentry {
  #define	LLE_ADDREF(lle) do {					\
  	LLE_WLOCK_ASSERT(lle);					\
  	KASSERT((lle)->lle_refcnt >= 0,				\
 -		("negative refcnt %d", (lle)->lle_refcnt));	\
 +	    ("negative refcnt %d on lle %p",			\
 +	    (lle)->lle_refcnt, (lle)));				\
  	(lle)->lle_refcnt++;					\
  } while (0)
  
  #define	LLE_REMREF(lle)	do {					\
  	LLE_WLOCK_ASSERT(lle);					\
 -	KASSERT((lle)->lle_refcnt > 1,				\
 -		("bogus refcnt %d", (lle)->lle_refcnt));	\
 +	KASSERT((lle)->lle_refcnt > 0,				\
 +	    ("bogus refcnt %d on lle %p",			\
 +	    (lle)->lle_refcnt, (lle)));				\
  	(lle)->lle_refcnt--;					\
  } while (0)
  
  #define	LLE_FREE_LOCKED(lle) do {				\
 -	if ((lle)->lle_refcnt <= 1)				\
 -		(lle)->lle_free((lle)->lle_tbl, (lle));\
 +	if ((lle)->lle_refcnt == 1)				\
 +		(lle)->lle_free((lle)->lle_tbl, (lle));		\
  	else {							\
 -		(lle)->lle_refcnt--;				\
 +		LLE_REMREF(lle);				\
  		LLE_WUNLOCK(lle);				\
  	}							\
  	/* guard against invalid refs */			\
 -	lle = NULL;						\
 +	(lle) = NULL;						\
  } while (0)
  
  #define	LLE_FREE(lle) do {					\
 @@ -172,6 +174,7 @@ MALLOC_DECLARE(M_LLTABLE);
  #define	LLE_VALID	0x0008	/* ll_addr is valid */
  #define	LLE_PROXY	0x0010	/* proxy entry ??? */
  #define	LLE_PUB		0x0020	/* publish entry ??? */
 +#define	LLE_LINKED	0x0040	/* linked to lookup structure */
  #define	LLE_EXCLUSIVE	0x2000	/* return lle xlocked  */
  #define	LLE_DELETE	0x4000	/* delete on a lookup - match LLE_IFADDR */
  #define	LLE_CREATE	0x8000	/* create on a lookup miss */
 
 Modified: head/sys/net/if_var.h
 ==============================================================================
 --- head/sys/net/if_var.h	Thu Aug  2 13:20:44 2012	(r238989)
 +++ head/sys/net/if_var.h	Thu Aug  2 13:57:49 2012	(r238990)
 @@ -415,6 +415,8 @@ EVENTHANDLER_DECLARE(group_change_event,
  #define	IF_AFDATA_DESTROY(ifp)	rw_destroy(&(ifp)->if_afdata_lock)
  
  #define	IF_AFDATA_LOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_LOCKED)
 +#define	IF_AFDATA_RLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_RLOCKED)
 +#define	IF_AFDATA_WLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_WLOCKED)
  #define	IF_AFDATA_UNLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_UNLOCKED)
  
  int	if_handoff(struct ifqueue *ifq, struct mbuf *m, struct ifnet *ifp,
 
 Modified: head/sys/netinet/if_ether.c
 ==============================================================================
 --- head/sys/netinet/if_ether.c	Thu Aug  2 13:20:44 2012	(r238989)
 +++ head/sys/netinet/if_ether.c	Thu Aug  2 13:57:49 2012	(r238990)
 @@ -163,49 +163,40 @@ arp_ifscrub(struct ifnet *ifp, uint32_t 
  static void
  arptimer(void *arg)
  {
 +	struct llentry *lle = (struct llentry *)arg;
  	struct ifnet *ifp;
 -	struct llentry   *lle;
 -	int pkts_dropped;
 +	size_t pkts_dropped;
 +
 +	if (lle->la_flags & LLE_STATIC) {
 +		LLE_WUNLOCK(lle);
 +		return;
 +	}
  
 -	KASSERT(arg != NULL, ("%s: arg NULL", __func__));
 -	lle = (struct llentry *)arg;
  	ifp = lle->lle_tbl->llt_ifp;
  	CURVNET_SET(ifp->if_vnet);
 +
 +	if (lle->la_flags != LLE_DELETED) {
 +		int evt;
 +
 +		if (lle->la_flags & LLE_VALID)
 +			evt = LLENTRY_EXPIRED;
 +		else
 +			evt = LLENTRY_TIMEDOUT;
 +		EVENTHANDLER_INVOKE(lle_event, lle, evt);
 +	}
 +
 +	callout_stop(&lle->la_timer);
 +
 +	/* XXX: LOR avoidance. We still have ref on lle. */
 +	LLE_WUNLOCK(lle);
  	IF_AFDATA_LOCK(ifp);
  	LLE_WLOCK(lle);
 -	if (lle->la_flags & LLE_STATIC)
 -		LLE_WUNLOCK(lle);
 -	else {
 -		if (!callout_pending(&lle->la_timer) &&
 -		    callout_active(&lle->la_timer)) {
 -			callout_stop(&lle->la_timer);
 -			LLE_REMREF(lle);
 -
 -			if (lle->la_flags != LLE_DELETED) {
 -				int evt;
 -
 -				if (lle->la_flags & LLE_VALID)
 -					evt = LLENTRY_EXPIRED;
 -				else
 -					evt = LLENTRY_TIMEDOUT;
 -				EVENTHANDLER_INVOKE(lle_event, lle, evt);
 -			}
  
 -			pkts_dropped = llentry_free(lle);
 -			ARPSTAT_ADD(dropped, pkts_dropped);
 -			ARPSTAT_INC(timeouts);
 -		} else {
 -#ifdef DIAGNOSTIC
 -			struct sockaddr *l3addr = L3_ADDR(lle);
 -			log(LOG_INFO,
 -			    "arptimer issue: %p, IPv4 address: \"%s\"\n", lle,
 -			    inet_ntoa(
 -			        ((const struct sockaddr_in *)l3addr)->sin_addr));
 -#endif
 -			LLE_WUNLOCK(lle);
 -		}
 -	}
 +	LLE_REMREF(lle);
 +	pkts_dropped = llentry_free(lle);
  	IF_AFDATA_UNLOCK(ifp);
 +	ARPSTAT_ADD(dropped, pkts_dropped);
 +	ARPSTAT_INC(timeouts);
  	CURVNET_RESTORE();
  }
  
 
 Modified: head/sys/netinet/in.c
 ==============================================================================
 --- head/sys/netinet/in.c	Thu Aug  2 13:20:44 2012	(r238989)
 +++ head/sys/netinet/in.c	Thu Aug  2 13:57:49 2012	(r238990)
 @@ -1280,7 +1280,6 @@ in_lltable_new(const struct sockaddr *l3
  	if (lle == NULL)		/* NB: caller generates msg */
  		return NULL;
  
 -	callout_init(&lle->base.la_timer, CALLOUT_MPSAFE);
  	/*
  	 * For IPv4 this will trigger "arpresolve" to generate
  	 * an ARP request.
 @@ -1290,7 +1289,10 @@ in_lltable_new(const struct sockaddr *l3
  	lle->base.lle_refcnt = 1;
  	lle->base.lle_free = in_lltable_free;
  	LLE_LOCK_INIT(&lle->base);
 -	return &lle->base;
 +	callout_init_rw(&lle->base.la_timer, &lle->base.lle_lock,
 +	    CALLOUT_RETURNUNLOCKED);
 +
 +	return (&lle->base);
  }
  
  #define IN_ARE_MASKED_ADDR_EQUAL(d, a, m)	(			\
 @@ -1306,6 +1308,7 @@ in_lltable_prefix_free(struct lltable *l
  	int i;
  	size_t pkts_dropped;
  
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
  	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  			/*
 @@ -1315,17 +1318,15 @@ in_lltable_prefix_free(struct lltable *l
  			if (IN_ARE_MASKED_ADDR_EQUAL(satosin(L3_ADDR(lle)),
  			    pfx, msk) && ((flags & LLE_STATIC) ||
  			    !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 -
 -				canceled = callout_drain(&lle->la_timer);
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				pkts_dropped = llentry_free(lle);
  				ARPSTAT_ADD(dropped, pkts_dropped);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  
 @@ -1457,11 +1458,12 @@ in_lltable_lookup(struct lltable *llt, u
  
  		lle->lle_tbl  = llt;
  		lle->lle_head = lleh;
 +		lle->la_flags |= LLE_LINKED;
  		LIST_INSERT_HEAD(lleh, lle, lle_next);
  	} else if (flags & LLE_DELETE) {
  		if (!(lle->la_flags & LLE_IFADDR) || (flags & LLE_IFADDR)) {
  			LLE_WLOCK(lle);
 -			lle->la_flags = LLE_DELETED;
 +			lle->la_flags |= LLE_DELETED;
  			EVENTHANDLER_INVOKE(lle_event, lle, LLENTRY_DELETED);
  			LLE_WUNLOCK(lle);
  #ifdef DIAGNOSTIC
 
 Modified: head/sys/netinet6/in6.c
 ==============================================================================
 --- head/sys/netinet6/in6.c	Thu Aug  2 13:20:44 2012	(r238989)
 +++ head/sys/netinet6/in6.c	Thu Aug  2 13:57:49 2012	(r238990)
 @@ -2497,23 +2497,22 @@ in6_lltable_prefix_free(struct lltable *
  	 * (flags & LLE_STATIC) means deleting all entries
  	 * including static ND6 entries.
  	 */
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
  	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  			if (IN6_ARE_MASKED_ADDR_EQUAL(
 -				    &((struct sockaddr_in6 *)L3_ADDR(lle))->sin6_addr,
 -				    &pfx->sin6_addr,
 -				    &msk->sin6_addr) &&
 -			    ((flags & LLE_STATIC) || !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 -
 -				canceled = callout_drain(&lle->la_timer);
 +			    &satosin6(L3_ADDR(lle))->sin6_addr,
 +			    &pfx->sin6_addr, &msk->sin6_addr) &&
 +			    ((flags & LLE_STATIC) ||
 +			    !(lle->la_flags & LLE_STATIC))) {
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				llentry_free(lle);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  static int
 @@ -2605,11 +2604,12 @@ in6_lltable_lookup(struct lltable *llt, 
  
  		lle->lle_tbl  = llt;
  		lle->lle_head = lleh;
 +		lle->la_flags |= LLE_LINKED;
  		LIST_INSERT_HEAD(lleh, lle, lle_next);
  	} else if (flags & LLE_DELETE) {
  		if (!(lle->la_flags & LLE_IFADDR) || (flags & LLE_IFADDR)) {
  			LLE_WLOCK(lle);
 -			lle->la_flags = LLE_DELETED;
 +			lle->la_flags |= LLE_DELETED;
  			LLE_WUNLOCK(lle);
  #ifdef DIAGNOSTIC
  			log(LOG_INFO, "ifaddr cache = %p  is deleted\n", lle);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->patched 
State-Changed-By: glebius 
State-Changed-When: Fri Aug 3 08:01:48 UTC 2012 
State-Changed-Why:  
Fix committed to head/. 


Responsible-Changed-From-To: freebsd-net->glebius 
Responsible-Changed-By: glebius 
Responsible-Changed-When: Fri Aug 3 08:01:48 UTC 2012 
Responsible-Changed-Why:  
Fix committed to head/. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165863 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/165863: commit references a PR
Date: Mon, 10 Sep 2012 12:26:21 +0000 (UTC)

 Author: glebius
 Date: Mon Sep 10 12:25:57 2012
 New Revision: 240313
 URL: http://svn.freebsd.org/changeset/base/240313
 
 Log:
   Merge r238990 (manually resolving absence of r237263):
     Fix races between in_lltable_prefix_free(), lla_lookup(),
     llentry_free() and arptimer():
   
     o Use callout_init_rw() for lle timeout, this allows us safely
       disestablish them.
       - This allows us to simplify the arptimer() and make it
         race safe.
     o Consistently use ifp->if_afdata_lock to lock access to
       linked lists in the lle hashes.
     o Introduce new lle flag LLE_LINKED, which marks an entry that
       is attached to the hash.
       - Use LLE_LINKED to avoid double unlinking via consequent
         calls to llentry_free().
       - Mark lle with LLE_DELETED via |= operation istead of =,
         so that other flags won't be lost.
     o Make LLE_ADDREF(), LLE_REMREF() and LLE_FREE_LOCKED() more
       consistent and provide more informative KASSERTs.
   
     The patch is a collaborative work of all submitters and myself.
   
     PR:		kern/165863
     Submitted by:	zont, rstone
     Submitted by:	Eric van Gyzen <eric_van_gyzen dell.com>
 
 Modified:
   stable/9/sys/net/if_llatbl.c
   stable/9/sys/net/if_llatbl.h
   stable/9/sys/net/if_var.h
   stable/9/sys/netinet/if_ether.c
   stable/9/sys/netinet/in.c
   stable/9/sys/netinet6/in6.c
 Directory Properties:
   stable/9/sys/   (props changed)
 
 Modified: stable/9/sys/net/if_llatbl.c
 ==============================================================================
 --- stable/9/sys/net/if_llatbl.c	Mon Sep 10 12:23:56 2012	(r240312)
 +++ stable/9/sys/net/if_llatbl.c	Mon Sep 10 12:25:57 2012	(r240313)
 @@ -109,10 +109,19 @@ llentry_free(struct llentry *lle)
  	size_t pkts_dropped;
  	struct mbuf *next;
  
 -	pkts_dropped = 0;
 +	IF_AFDATA_WLOCK_ASSERT(lle->lle_tbl->llt_ifp);
  	LLE_WLOCK_ASSERT(lle);
 +
 +	/* XXX: guard against race with other llentry_free(). */
 +	if (!(lle->la_flags & LLE_LINKED)) {
 +		LLE_FREE_LOCKED(lle);
 +		return (0);
 +	}
 +
  	LIST_REMOVE(lle, lle_next);
 +	lle->la_flags &= ~(LLE_VALID | LLE_LINKED);
  
 +	pkts_dropped = 0;
  	while ((lle->la_numheld > 0) && (lle->la_hold != NULL)) {
  		next = lle->la_hold->m_nextpkt;
  		m_freem(lle->la_hold);
 @@ -125,7 +134,6 @@ llentry_free(struct llentry *lle)
  		("%s: la_numheld %d > 0, pkts_droped %zd", __func__,
  		 lle->la_numheld, pkts_dropped));
  
 -	lle->la_flags &= ~LLE_VALID;
  	LLE_FREE_LOCKED(lle);
  
  	return (pkts_dropped);
 @@ -176,17 +184,16 @@ lltable_free(struct lltable *llt)
  	SLIST_REMOVE(&V_lltables, llt, lltable, llt_link);
  	LLTABLE_WUNLOCK();
  
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
  	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
 -			int canceled;
 -
 -			canceled = callout_drain(&lle->la_timer);
  			LLE_WLOCK(lle);
 -			if (canceled)
 +			if (callout_stop(&lle->la_timer))
  				LLE_REMREF(lle);
  			llentry_free(lle);
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  
  	free(llt, M_LLTABLE);
  }
 
 Modified: stable/9/sys/net/if_llatbl.h
 ==============================================================================
 --- stable/9/sys/net/if_llatbl.h	Mon Sep 10 12:23:56 2012	(r240312)
 +++ stable/9/sys/net/if_llatbl.h	Mon Sep 10 12:25:57 2012	(r240313)
 @@ -103,26 +103,28 @@ struct llentry {
  #define	LLE_ADDREF(lle) do {					\
  	LLE_WLOCK_ASSERT(lle);					\
  	KASSERT((lle)->lle_refcnt >= 0,				\
 -		("negative refcnt %d", (lle)->lle_refcnt));	\
 +	    ("negative refcnt %d on lle %p",			\
 +	    (lle)->lle_refcnt, (lle)));				\
  	(lle)->lle_refcnt++;					\
  } while (0)
  
  #define	LLE_REMREF(lle)	do {					\
  	LLE_WLOCK_ASSERT(lle);					\
 -	KASSERT((lle)->lle_refcnt > 1,				\
 -		("bogus refcnt %d", (lle)->lle_refcnt));	\
 +	KASSERT((lle)->lle_refcnt > 0,				\
 +	    ("bogus refcnt %d on lle %p",			\
 +	    (lle)->lle_refcnt, (lle)));				\
  	(lle)->lle_refcnt--;					\
  } while (0)
  
  #define	LLE_FREE_LOCKED(lle) do {				\
 -	if ((lle)->lle_refcnt <= 1)				\
 -		(lle)->lle_free((lle)->lle_tbl, (lle));\
 +	if ((lle)->lle_refcnt == 1)				\
 +		(lle)->lle_free((lle)->lle_tbl, (lle));		\
  	else {							\
 -		(lle)->lle_refcnt--;				\
 +		LLE_REMREF(lle);				\
  		LLE_WUNLOCK(lle);				\
  	}							\
  	/* guard against invalid refs */			\
 -	lle = NULL;						\
 +	(lle) = NULL;						\
  } while (0)
  
  #define	LLE_FREE(lle) do {					\
 @@ -172,6 +174,7 @@ MALLOC_DECLARE(M_LLTABLE);
  #define	LLE_VALID	0x0008	/* ll_addr is valid */
  #define	LLE_PROXY	0x0010	/* proxy entry ??? */
  #define	LLE_PUB		0x0020	/* publish entry ??? */
 +#define	LLE_LINKED	0x0040	/* linked to lookup structure */
  #define	LLE_EXCLUSIVE	0x2000	/* return lle xlocked  */
  #define	LLE_DELETE	0x4000	/* delete on a lookup - match LLE_IFADDR */
  #define	LLE_CREATE	0x8000	/* create on a lookup miss */
 
 Modified: stable/9/sys/net/if_var.h
 ==============================================================================
 --- stable/9/sys/net/if_var.h	Mon Sep 10 12:23:56 2012	(r240312)
 +++ stable/9/sys/net/if_var.h	Mon Sep 10 12:25:57 2012	(r240313)
 @@ -419,6 +419,8 @@ EVENTHANDLER_DECLARE(group_change_event,
  #define	IF_AFDATA_DESTROY(ifp)	rw_destroy(&(ifp)->if_afdata_lock)
  
  #define	IF_AFDATA_LOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_LOCKED)
 +#define	IF_AFDATA_RLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_RLOCKED)
 +#define	IF_AFDATA_WLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_WLOCKED)
  #define	IF_AFDATA_UNLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_UNLOCKED)
  
  int	if_handoff(struct ifqueue *ifq, struct mbuf *m, struct ifnet *ifp,
 
 Modified: stable/9/sys/netinet/if_ether.c
 ==============================================================================
 --- stable/9/sys/netinet/if_ether.c	Mon Sep 10 12:23:56 2012	(r240312)
 +++ stable/9/sys/netinet/if_ether.c	Mon Sep 10 12:25:57 2012	(r240313)
 @@ -167,38 +167,30 @@ arp_ifscrub(struct ifnet *ifp, uint32_t 
  static void
  arptimer(void *arg)
  {
 +	struct llentry *lle = (struct llentry *)arg;
  	struct ifnet *ifp;
 -	struct llentry   *lle;
 -	int pkts_dropped;
 +	size_t pkts_dropped;
 +
 +	if (lle->la_flags & LLE_STATIC) {
 +		LLE_WUNLOCK(lle);
 +		return;
 +	}
  
 -	KASSERT(arg != NULL, ("%s: arg NULL", __func__));
 -	lle = (struct llentry *)arg;
  	ifp = lle->lle_tbl->llt_ifp;
  	CURVNET_SET(ifp->if_vnet);
 +
 +	callout_stop(&lle->la_timer);
 +
 +	/* XXX: LOR avoidance. We still have ref on lle. */
 +	LLE_WUNLOCK(lle);
  	IF_AFDATA_LOCK(ifp);
  	LLE_WLOCK(lle);
 -	if (lle->la_flags & LLE_STATIC)
 -		LLE_WUNLOCK(lle);
 -	else {
 -		if (!callout_pending(&lle->la_timer) &&
 -		    callout_active(&lle->la_timer)) {
 -			callout_stop(&lle->la_timer);
 -			LLE_REMREF(lle);
 -			pkts_dropped = llentry_free(lle);
 -			ARPSTAT_ADD(dropped, pkts_dropped);
 -			ARPSTAT_INC(timeouts);
 -		} else {
 -#ifdef DIAGNOSTIC
 -			struct sockaddr *l3addr = L3_ADDR(lle);
 -			log(LOG_INFO,
 -			    "arptimer issue: %p, IPv4 address: \"%s\"\n", lle,
 -			    inet_ntoa(
 -			        ((const struct sockaddr_in *)l3addr)->sin_addr));
 -#endif
 -			LLE_WUNLOCK(lle);
 -		}
 -	}
 +
 +	LLE_REMREF(lle);
 +	pkts_dropped = llentry_free(lle);
  	IF_AFDATA_UNLOCK(ifp);
 +	ARPSTAT_ADD(dropped, pkts_dropped);
 +	ARPSTAT_INC(timeouts);
  	CURVNET_RESTORE();
  }
  
 
 Modified: stable/9/sys/netinet/in.c
 ==============================================================================
 --- stable/9/sys/netinet/in.c	Mon Sep 10 12:23:56 2012	(r240312)
 +++ stable/9/sys/netinet/in.c	Mon Sep 10 12:25:57 2012	(r240313)
 @@ -1343,7 +1343,6 @@ in_lltable_new(const struct sockaddr *l3
  	if (lle == NULL)		/* NB: caller generates msg */
  		return NULL;
  
 -	callout_init(&lle->base.la_timer, CALLOUT_MPSAFE);
  	/*
  	 * For IPv4 this will trigger "arpresolve" to generate
  	 * an ARP request.
 @@ -1353,7 +1352,10 @@ in_lltable_new(const struct sockaddr *l3
  	lle->base.lle_refcnt = 1;
  	lle->base.lle_free = in_lltable_free;
  	LLE_LOCK_INIT(&lle->base);
 -	return &lle->base;
 +	callout_init_rw(&lle->base.la_timer, &lle->base.lle_lock,
 +	    CALLOUT_RETURNUNLOCKED);
 +
 +	return (&lle->base);
  }
  
  #define IN_ARE_MASKED_ADDR_EQUAL(d, a, m)	(			\
 @@ -1369,6 +1371,7 @@ in_lltable_prefix_free(struct lltable *l
  	int i;
  	size_t pkts_dropped;
  
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
  	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  			/*
 @@ -1378,17 +1381,15 @@ in_lltable_prefix_free(struct lltable *l
  			if (IN_ARE_MASKED_ADDR_EQUAL(satosin(L3_ADDR(lle)),
  			    pfx, msk) && ((flags & LLE_STATIC) ||
  			    !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 -
 -				canceled = callout_drain(&lle->la_timer);
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				pkts_dropped = llentry_free(lle);
  				ARPSTAT_ADD(dropped, pkts_dropped);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  
 @@ -1520,11 +1521,12 @@ in_lltable_lookup(struct lltable *llt, u
  
  		lle->lle_tbl  = llt;
  		lle->lle_head = lleh;
 +		lle->la_flags |= LLE_LINKED;
  		LIST_INSERT_HEAD(lleh, lle, lle_next);
  	} else if (flags & LLE_DELETE) {
  		if (!(lle->la_flags & LLE_IFADDR) || (flags & LLE_IFADDR)) {
  			LLE_WLOCK(lle);
 -			lle->la_flags = LLE_DELETED;
 +			lle->la_flags |= LLE_DELETED;
  			EVENTHANDLER_INVOKE(arp_update_event, lle);
  			LLE_WUNLOCK(lle);
  #ifdef DIAGNOSTIC
 
 Modified: stable/9/sys/netinet6/in6.c
 ==============================================================================
 --- stable/9/sys/netinet6/in6.c	Mon Sep 10 12:23:56 2012	(r240312)
 +++ stable/9/sys/netinet6/in6.c	Mon Sep 10 12:25:57 2012	(r240313)
 @@ -2468,23 +2468,22 @@ in6_lltable_prefix_free(struct lltable *
  	 * (flags & LLE_STATIC) means deleting all entries
  	 * including static ND6 entries.
  	 */
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
  	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  			if (IN6_ARE_MASKED_ADDR_EQUAL(
 -				    &((struct sockaddr_in6 *)L3_ADDR(lle))->sin6_addr,
 -				    &pfx->sin6_addr,
 -				    &msk->sin6_addr) &&
 -			    ((flags & LLE_STATIC) || !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 -
 -				canceled = callout_drain(&lle->la_timer);
 +			    &satosin6(L3_ADDR(lle))->sin6_addr,
 +			    &pfx->sin6_addr, &msk->sin6_addr) &&
 +			    ((flags & LLE_STATIC) ||
 +			    !(lle->la_flags & LLE_STATIC))) {
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				llentry_free(lle);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  static int
 @@ -2576,11 +2575,12 @@ in6_lltable_lookup(struct lltable *llt, 
  
  		lle->lle_tbl  = llt;
  		lle->lle_head = lleh;
 +		lle->la_flags |= LLE_LINKED;
  		LIST_INSERT_HEAD(lleh, lle, lle_next);
  	} else if (flags & LLE_DELETE) {
  		if (!(lle->la_flags & LLE_IFADDR) || (flags & LLE_IFADDR)) {
  			LLE_WLOCK(lle);
 -			lle->la_flags = LLE_DELETED;
 +			lle->la_flags |= LLE_DELETED;
  			LLE_WUNLOCK(lle);
  #ifdef DIAGNOSTIC
  			log(LOG_INFO, "ifaddr cache = %p  is deleted\n", lle);
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: patched->closed 
State-Changed-By: glebius 
State-Changed-When: Mon Sep 10 12:31:09 UTC 2012 
State-Changed-Why:  
Merged to stable/9. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=165863 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/165863: commit references a PR
Date: Wed, 11 Sep 2013 07:24:56 +0000 (UTC)

 Author: dteske
 Date: Wed Sep 11 07:24:46 2013
 New Revision: 255470
 URL: http://svnweb.freebsd.org/changeset/base/255470
 
 Log:
   Merge from stable/9 SVN r250927: MFC: r249628, r249742
   
   - recover missing arp_ifinit() call.
   - plug static llentry leak (ipv4 & ipv6 were affected).
   
   PR:		kern/172985
   
   Also merge from stable/9 SVN r240313:
   
   Merge r238990 (manually resolving absence of r237263):
     Fix races between in_lltable_prefix_free(), lla_lookup(),
     llentry_free() and arptimer():
   
     o Use callout_init_rw() for lle timeout, this allows us safely
       disestablish them.
       - This allows us to simplify the arptimer() and make it
         race safe.
     o Consistently use ifp->if_afdata_lock to lock access to
       linked lists in the lle hashes.
     o Introduce new lle flag LLE_LINKED, which marks an entry that
       is attached to the hash.
       - Use LLE_LINKED to avoid double unlinking via consequent
         calls to llentry_free().
       - Mark lle with LLE_DELETED via |= operation istead of =,
         so that other flags won't be lost.
     o Make LLE_ADDREF(), LLE_REMREF() and LLE_FREE_LOCKED() more
       consistent and provide more informative KASSERTs.
   
     The patch is a collaborative work of all submitters and myself.
   
     PR:		kern/165863
     Submitted by:	zont, rstone
     Submitted by:	Eric van Gyzen <eric_van_gyzen dell.com>
 
 Modified:
   stable/8/sys/net/if_llatbl.c
   stable/8/sys/net/if_llatbl.h
   stable/8/sys/net/if_var.h
   stable/8/sys/net/if_vlan.c
   stable/8/sys/netinet/if_ether.c
   stable/8/sys/netinet/in.c
   stable/8/sys/netinet6/in6.c
 Directory Properties:
   stable/8/sys/   (props changed)
   stable/8/sys/net/   (props changed)
   stable/8/sys/netinet/   (props changed)
   stable/8/sys/netinet6/   (props changed)
 
 Modified: stable/8/sys/net/if_llatbl.c
 ==============================================================================
 --- stable/8/sys/net/if_llatbl.c	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/net/if_llatbl.c	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -109,10 +109,19 @@ llentry_free(struct llentry *lle)
  	size_t pkts_dropped;
  	struct mbuf *next;
  
 -	pkts_dropped = 0;
 +	IF_AFDATA_WLOCK_ASSERT(lle->lle_tbl->llt_ifp);
  	LLE_WLOCK_ASSERT(lle);
 +
 +	/* XXX: guard against race with other llentry_free(). */
 +	if (!(lle->la_flags & LLE_LINKED)) {
 +		LLE_FREE_LOCKED(lle);
 +		return (0);
 +	}
 +
  	LIST_REMOVE(lle, lle_next);
 +	lle->la_flags &= ~(LLE_VALID | LLE_LINKED);
  
 +	pkts_dropped = 0;
  	while ((lle->la_numheld > 0) && (lle->la_hold != NULL)) {
  		next = lle->la_hold->m_nextpkt;
  		m_freem(lle->la_hold);
 @@ -125,7 +134,6 @@ llentry_free(struct llentry *lle)
  		("%s: la_numheld %d > 0, pkts_droped %zd", __func__, 
  		 lle->la_numheld, pkts_dropped));
  
 -	lle->la_flags &= ~LLE_VALID;
  	LLE_FREE_LOCKED(lle);
  
  	return (pkts_dropped);
 @@ -185,17 +193,16 @@ lltable_free(struct lltable *llt)
  	SLIST_REMOVE(&V_lltables, llt, lltable, llt_link);
  	LLTABLE_WUNLOCK();
  
 -	for (i=0; i < LLTBL_HASHTBL_SIZE; i++) {
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
 +	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
 -			int canceled;
 -
 -			canceled = callout_drain(&lle->la_timer);
  			LLE_WLOCK(lle);
 -			if (canceled)
 +			if (callout_stop(&lle->la_timer))
  				LLE_REMREF(lle);
  			llentry_free(lle);
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  
  	free(llt, M_LLTABLE);
  }
 
 Modified: stable/8/sys/net/if_llatbl.h
 ==============================================================================
 --- stable/8/sys/net/if_llatbl.h	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/net/if_llatbl.h	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -97,26 +97,28 @@ struct llentry {
  #define	LLE_ADDREF(lle) do {					\
  	LLE_WLOCK_ASSERT(lle);					\
  	KASSERT((lle)->lle_refcnt >= 0,				\
 -		("negative refcnt %d", (lle)->lle_refcnt));	\
 +	    ("negative refcnt %d on lle %p",			\
 +	    (lle)->lle_refcnt, (lle)));				\
  	(lle)->lle_refcnt++;					\
  } while (0)
  
  #define	LLE_REMREF(lle)	do {					\
  	LLE_WLOCK_ASSERT(lle);					\
 -	KASSERT((lle)->lle_refcnt > 1,				\
 -		("bogus refcnt %d", (lle)->lle_refcnt));	\
 +	KASSERT((lle)->lle_refcnt > 0,				\
 +	    ("bogus refcnt %d on lle %p",			\
 +	    (lle)->lle_refcnt, (lle)));				\
  	(lle)->lle_refcnt--;					\
  } while (0)
  
  #define	LLE_FREE_LOCKED(lle) do {				\
 -	if ((lle)->lle_refcnt <= 1)				\
 +	if ((lle)->lle_refcnt == 1)				\
  		(lle)->lle_tbl->llt_free((lle)->lle_tbl, (lle));\
  	else {							\
 -		(lle)->lle_refcnt--;				\
 +		LLE_REMREF(lle);				\
  		LLE_WUNLOCK(lle);				\
  	}							\
  	/* guard against invalid refs */			\
 -	lle = NULL;						\
 +	(lle) = NULL;						\
  } while (0)
  
  #define	LLE_FREE(lle) do {					\
 @@ -167,9 +169,10 @@ MALLOC_DECLARE(M_LLTABLE);
  #define	LLE_VALID	0x0008	/* ll_addr is valid */
  #define	LLE_PROXY	0x0010	/* proxy entry ??? */
  #define	LLE_PUB		0x0020	/* publish entry ??? */
 +#define	LLE_LINKED	0x0040	/* linked to lookup structure */
 +#define	LLE_EXCLUSIVE	0x2000	/* return lle xlocked  */
  #define	LLE_DELETE	0x4000	/* delete on a lookup - match LLE_IFADDR */
  #define	LLE_CREATE	0x8000	/* create on a lookup miss */
 -#define	LLE_EXCLUSIVE	0x2000	/* return lle xlocked  */
  
  #define LLATBL_HASH(key, mask) \
  	(((((((key >> 8) ^ key) >> 8) ^ key) >> 8) ^ key) & mask)
 
 Modified: stable/8/sys/net/if_var.h
 ==============================================================================
 --- stable/8/sys/net/if_var.h	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/net/if_var.h	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -406,6 +406,8 @@ EVENTHANDLER_DECLARE(group_change_event,
  #define	IF_AFDATA_DESTROY(ifp)	rw_destroy(&(ifp)->if_afdata_lock)
  
  #define	IF_AFDATA_LOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_LOCKED)
 +#define	IF_AFDATA_RLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_RLOCKED)
 +#define	IF_AFDATA_WLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_WLOCKED)
  #define	IF_AFDATA_UNLOCK_ASSERT(ifp)	rw_assert(&(ifp)->if_afdata_lock, RA_UNLOCKED)
  
  int	if_handoff(struct ifqueue *ifq, struct mbuf *m, struct ifnet *ifp,
 
 Modified: stable/8/sys/net/if_vlan.c
 ==============================================================================
 --- stable/8/sys/net/if_vlan.c	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/net/if_vlan.c	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -41,6 +41,7 @@
  #include <sys/cdefs.h>
  __FBSDID("$FreeBSD$");
  
 +#include "opt_inet.h"
  #include "opt_vlan.h"
  
  #include <sys/param.h>
 @@ -65,6 +66,11 @@ __FBSDID("$FreeBSD$");
  #include <net/if_vlan_var.h>
  #include <net/vnet.h>
  
 +#ifdef INET
 +#include <netinet/in.h>
 +#include <netinet/if_ether.h>
 +#endif
 +
  #define VLANNAME	"vlan"
  #define	VLAN_DEF_HWIDTH	4
  #define	VLAN_IFFLAGS	(IFF_BROADCAST | IFF_MULTICAST)
 
 Modified: stable/8/sys/netinet/if_ether.c
 ==============================================================================
 --- stable/8/sys/netinet/if_ether.c	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/netinet/if_ether.c	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -167,38 +167,30 @@ arp_ifscrub(struct ifnet *ifp, uint32_t 
  static void
  arptimer(void *arg)
  {
 +	struct llentry *lle = (struct llentry *)arg;
  	struct ifnet *ifp;
 -	struct llentry   *lle;
 -	int pkts_dropped;
 +	size_t pkts_dropped;
 +
 +	if (lle->la_flags & LLE_STATIC) {
 +		LLE_WUNLOCK(lle);
 +		return;
 +	}
  
 -	KASSERT(arg != NULL, ("%s: arg NULL", __func__));
 -	lle = (struct llentry *)arg;
  	ifp = lle->lle_tbl->llt_ifp;
  	CURVNET_SET(ifp->if_vnet);
 +
 +	callout_stop(&lle->la_timer);
 +
 +	/* XXX: LOR avoidance. We still have ref on lle. */
 +	LLE_WUNLOCK(lle);
  	IF_AFDATA_LOCK(ifp);
  	LLE_WLOCK(lle);
 -	if (lle->la_flags & LLE_STATIC)
 -		LLE_WUNLOCK(lle);
 -	else {
 -		if (!callout_pending(&lle->la_timer) &&
 -		    callout_active(&lle->la_timer)) {
 -			callout_stop(&lle->la_timer);
 -			LLE_REMREF(lle);
 -			pkts_dropped = llentry_free(lle);
 -			ARPSTAT_ADD(dropped, pkts_dropped);
 -			ARPSTAT_INC(timeouts);
 -		} else {
 -#ifdef DIAGNOSTIC
 -			struct sockaddr *l3addr = L3_ADDR(lle);
 -			log(LOG_INFO, 
 -			    "arptimer issue: %p, IPv4 address: \"%s\"\n", lle,
 -			    inet_ntoa(
 -			        ((const struct sockaddr_in *)l3addr)->sin_addr));
 -#endif
 -			LLE_WUNLOCK(lle);
 -		}
 -	}
 +
 +	LLE_REMREF(lle);
 +	pkts_dropped = llentry_free(lle);
  	IF_AFDATA_UNLOCK(ifp);
 +	ARPSTAT_ADD(dropped, pkts_dropped);
 +	ARPSTAT_INC(timeouts);
  	CURVNET_RESTORE();
  }
  
 
 Modified: stable/8/sys/netinet/in.c
 ==============================================================================
 --- stable/8/sys/netinet/in.c	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/netinet/in.c	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -1350,7 +1350,6 @@ in_lltable_new(const struct sockaddr *l3
  	if (lle == NULL)		/* NB: caller generates msg */
  		return NULL;
  
 -	callout_init(&lle->base.la_timer, CALLOUT_MPSAFE);
  	/*
  	 * For IPv4 this will trigger "arpresolve" to generate
  	 * an ARP request.
 @@ -1359,7 +1358,10 @@ in_lltable_new(const struct sockaddr *l3
  	lle->l3_addr4 = *(const struct sockaddr_in *)l3addr;
  	lle->base.lle_refcnt = 1;
  	LLE_LOCK_INIT(&lle->base);
 -	return &lle->base;
 +	callout_init_rw(&lle->base.la_timer, &lle->base.lle_lock,
 +	    CALLOUT_RETURNUNLOCKED);
 +
 +	return (&lle->base);
  }
  
  /*
 @@ -1392,7 +1394,8 @@ in_lltable_prefix_free(struct lltable *l
  	register int i;
  	size_t pkts_dropped;
  
 -	for (i=0; i < LLTBL_HASHTBL_SIZE; i++) {
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
 +	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  
  		        /* 
 @@ -1402,17 +1405,15 @@ in_lltable_prefix_free(struct lltable *l
  			if (IN_ARE_MASKED_ADDR_EQUAL((struct sockaddr_in *)L3_ADDR(lle), 
  						     pfx, msk) &&
  			    ((flags & LLE_STATIC) || !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 -
 -				canceled = callout_drain(&lle->la_timer);
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				pkts_dropped = llentry_free(lle);
  				ARPSTAT_ADD(dropped, pkts_dropped);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  
 @@ -1545,15 +1546,20 @@ in_lltable_lookup(struct lltable *llt, u
  
  		lle->lle_tbl  = llt;
  		lle->lle_head = lleh;
 +		lle->la_flags |= LLE_LINKED;
  		LIST_INSERT_HEAD(lleh, lle, lle_next);
  	} else if (flags & LLE_DELETE) {
  		if (!(lle->la_flags & LLE_IFADDR) || (flags & LLE_IFADDR)) {
  			LLE_WLOCK(lle);
 -			lle->la_flags = LLE_DELETED;
 -			LLE_WUNLOCK(lle);
 +			lle->la_flags |= LLE_DELETED;
  #ifdef DIAGNOSTIC
 -			log(LOG_INFO, "ifaddr cache = %p  is deleted\n", lle);	
 +			log(LOG_INFO, "ifaddr cache = %p is deleted\n", lle);
  #endif
 +			if ((lle->la_flags &
 +			    (LLE_STATIC | LLE_IFADDR)) == LLE_STATIC)
 +				llentry_free(lle);
 +			else
 +				LLE_WUNLOCK(lle);
  		}
  		lle = (void *)-1;
  		
 
 Modified: stable/8/sys/netinet6/in6.c
 ==============================================================================
 --- stable/8/sys/netinet6/in6.c	Wed Sep 11 07:11:14 2013	(r255469)
 +++ stable/8/sys/netinet6/in6.c	Wed Sep 11 07:24:46 2013	(r255470)
 @@ -1377,10 +1377,10 @@ in6_purgeaddr(struct ifaddr *ifa)
  	nd6_dad_stop(ifa);
  
  	/* Remove local address entry from lltable. */
 -	IF_AFDATA_LOCK(ifp);
 -	lla_lookup(LLTABLE6(ifp), (LLE_DELETE | LLE_IFADDR),
 -	    (struct sockaddr *)&ia->ia_addr);
 -	IF_AFDATA_UNLOCK(ifp);
 +	memcpy(&addr, &ia->ia_addr, sizeof(ia->ia_addr));
 +	memcpy(&mask, &ia->ia_prefixmask, sizeof(ia->ia_prefixmask));
 +	lltable_prefix_free(AF_INET6, (struct sockaddr *)&addr,
 +		(struct sockaddr *)&mask, LLE_STATIC);
  
  	/*
  	 * initialize for rtmsg generation
 @@ -1393,8 +1393,6 @@ in6_purgeaddr(struct ifaddr *ifa)
  	/* */
  	bzero(&rt0, sizeof(rt0));
  	rt0.rt_gateway = (struct sockaddr *)&gateway;
 -	memcpy(&mask, &ia->ia_prefixmask, sizeof(ia->ia_prefixmask));
 -	memcpy(&addr, &ia->ia_addr, sizeof(ia->ia_addr));
  	rt_mask(&rt0) = (struct sockaddr *)&mask;
  	rt_key(&rt0) = (struct sockaddr *)&addr;
  	rt0.rt_flags = RTF_HOST | RTF_STATIC;
 @@ -2392,23 +2390,22 @@ in6_lltable_prefix_free(struct lltable *
  	 * (flags & LLE_STATIC) means deleting all entries 
  	 * including static ND6 entries
  	 */
 -	for (i=0; i < LLTBL_HASHTBL_SIZE; i++) {
 +	IF_AFDATA_WLOCK(llt->llt_ifp);
 +	for (i = 0; i < LLTBL_HASHTBL_SIZE; i++) {
  		LIST_FOREACH_SAFE(lle, &llt->lle_head[i], lle_next, next) {
  			if (IN6_ARE_MASKED_ADDR_EQUAL(
 -				    &((struct sockaddr_in6 *)L3_ADDR(lle))->sin6_addr, 
 -				    &pfx->sin6_addr, 
 -				    &msk->sin6_addr) &&
 -			    ((flags & LLE_STATIC) || !(lle->la_flags & LLE_STATIC))) {
 -				int canceled;
 -
 -				canceled = callout_drain(&lle->la_timer);
 +			    &satosin6(L3_ADDR(lle))->sin6_addr,
 +			    &pfx->sin6_addr, &msk->sin6_addr) &&
 +			    ((flags & LLE_STATIC) ||
 +			    !(lle->la_flags & LLE_STATIC))) {
  				LLE_WLOCK(lle);
 -				if (canceled)
 +				if (callout_stop(&lle->la_timer))
  					LLE_REMREF(lle);
  				llentry_free(lle);
  			}
  		}
  	}
 +	IF_AFDATA_WUNLOCK(llt->llt_ifp);
  }
  
  static int
 @@ -2500,15 +2497,20 @@ in6_lltable_lookup(struct lltable *llt, 
  
  		lle->lle_tbl  = llt;
  		lle->lle_head = lleh;
 +		lle->la_flags |= LLE_LINKED;
  		LIST_INSERT_HEAD(lleh, lle, lle_next);
  	} else if (flags & LLE_DELETE) {
  		if (!(lle->la_flags & LLE_IFADDR) || (flags & LLE_IFADDR)) {
  			LLE_WLOCK(lle);
 -			lle->la_flags = LLE_DELETED;
 -			LLE_WUNLOCK(lle);
 +			lle->la_flags |= LLE_DELETED;
  #ifdef DIAGNOSTIC
 -			log(LOG_INFO, "ifaddr cache = %p  is deleted\n", lle);	
 -#endif	
 +			log(LOG_INFO, "ifaddr cache = %p is deleted\n", lle);
 +#endif
 +			if ((lle->la_flags &
 +			    (LLE_STATIC | LLE_IFADDR)) == LLE_STATIC)
 +				llentry_free(lle);
 +			else
 +				LLE_WUNLOCK(lle);
  		}
  		lle = (void *)-1;
  	}
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: "Teske, Devin" <Devin.Teske@fisglobal.com>
To: "bug-followup@FreeBSD.org" <bug-followup@FreeBSD.org>,
        "eric_van_gyzen@dell.com" <eric_van_gyzen@dell.com>
Cc: Devin Teske <dteske@freebsd.org>
Subject: Re: kern/165863: [panic] [netinet] [patch] in_lltable_prefix_free()
 races with lla_lookup() and arptimer()
Date: Wed, 11 Sep 2013 07:31:36 +0000

 The original reporter reported this on 8.2 and said 8.1 was also effected.
 
 We were able to replicate it on 8.1-R-p{0,3,4,6,12,13}. We were also able
 to replicate with a post-8.4 stable/8 @ r25522{8,9} kernel.
 
 We merged stable/9 revisions 250927 and r240313 and re-tested.
 
 The sample program provided in this PR no longer ends in kernel panic
 under stable/8 @ r255470.
 
 Before patch, it would panic between 30 and 90 seconds.
 After patch, ran longer than 10 minutes (didn't run longer considering the
 test causes DoS on the network).
 --=20
 Devin
 
 _____________
 The information contained in this message is proprietary and/or confidentia=
 l. If you are not the intended recipient, please: (i) delete the message an=
 d all copies; (ii) do not disclose, distribute or use the message in any ma=
 nner; and (iii) notify the sender immediately. In addition, please be aware=
  that any message addressed to our domain is subject to archiving and revie=
 w by persons other than the intended recipient. Thank you.
>Unformatted:
