From nobody@FreeBSD.org  Fri Apr 13 21:43:37 2012
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id 87935106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 13 Apr 2012 21:43:37 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 7388D8FC0A
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 13 Apr 2012 21:43:37 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q3DLhbU7083650
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 13 Apr 2012 21:43:37 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id q3DLhb78083629;
	Fri, 13 Apr 2012 21:43:37 GMT
	(envelope-from nobody)
Message-Id: <201204132143.q3DLhb78083629@red.freebsd.org>
Date: Fri, 13 Apr 2012 21:43:37 GMT
From: Ryan Stone <rstone@FreeBSD.org>
To: freebsd-gnats-submit@FreeBSD.org
Subject: fbt provider does not destroy probes when module unloaded
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         166928
>Category:       kern
>Synopsis:       [dtrace] fbt provider does not destroy probes when module unloaded
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    markj
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Apr 13 21:50:10 UTC 2012
>Closed-Date:    Sun Dec 29 17:50:59 UTC 2013
>Last-Modified:  Sun Dec 29 17:50:59 UTC 2013
>Originator:     Ryan Stone
>Release:        8.2-RELEASE
>Organization:
>Environment:
>Description:
# kldload dtraceall
# kldload hwpmc
# dtrace -n 'fbt:hwpmc::'
# kldunload hwpmc
# dtrace -n 'fbt:hwpmc::'
* kernel panic *

The first invocation of dtrace(1) creates the fbt probes for the hwpmc module, and they are not destroyed when hwpmc is unloaded.

In Solaris and its derivatives this is done by dtrace_module_unloaded in dtrace.c.  The implementation of dtrace_module_unloaded() is a bit hackish though and could incur collateral damage -- it looks like if I unload foo.ko, all probes matching *:foo:*:* will be destroyed, not just fbt:foo:*:*.
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->gnn 
Responsible-Changed-By: rstone 
Responsible-Changed-When: Mon Apr 16 19:27:48 UTC 2012 
Responsible-Changed-Why:  
over to gnn 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166928 
State-Changed-From-To: open->feedback 
State-Changed-By: gnn 
State-Changed-When: Tue Jun 5 15:49:12 UTC 2012 
State-Changed-Why:  
I cannot reproduce this on HEAD as of 5 May 2012. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166928 

From: Ryan Stone <rysto32@gmail.com>
To: bug-followup@FreeBSD.org, rstone@FreeBSD.org
Cc:  
Subject: Re: kern/166928: [dtrace] fbt provider does not destroy probes when
 module unloaded
Date: Tue, 5 Jun 2012 22:21:20 -0400

 Oops, I got the reproduction scenario slightly wrong.  It should have
 been (note the -lv arguments to the second dtrace invocation):
 
 # kldload dtraceall
 # kldload hwpmc
 # dtrace -n 'fbt:hwpmc::'
 # kldunload hwpmc
 # dtrace -lvn 'fbt:hwpmc::'
 
 The panic is:
 #0  doadump (textdump=1)
     at /usr/home/rstone/freebsd/head_2/sys/kern/kern_shutdown.c:266
 266             if (textdump && textdump_pending) {
 (kgdb) #0  doadump (textdump=1)
     at /usr/home/rstone/freebsd/head_2/sys/kern/kern_shutdown.c:266
 #1  0xffffffff808b3819 in kern_reboot (howto=260)
     at /usr/home/rstone/freebsd/head_2/sys/kern/kern_shutdown.c:449
 #2  0xffffffff808b324a in panic (fmt=0x0)
     at /usr/home/rstone/freebsd/head_2/sys/kern/kern_shutdown.c:637
 #3  0xffffffff80bb4930 in trap_fatal (frame=0x9, eva=Variable "eva" is not avail
 able.
 )
     at /usr/home/rstone/freebsd/head_2/sys/amd64/amd64/trap.c:852
 #4  0xffffffff80bb50be in trap (frame=0xffffff8025f575e0)
     at /usr/home/rstone/freebsd/head_2/sys/amd64/amd64/trap.c:599
 #5  0xffffffff80b9f5d3 in calltrap ()
     at /usr/home/rstone/freebsd/head_2/sys/amd64/amd64/exception.S:228
 #6  0xffffffff80893f31 in linker_ctf_get (file=0xfffffe000563a400,
     lc=0xffffff8025f57790)
     at /usr/home/rstone/freebsd/head_2/sys/kern/kern_linker.c:711#7
 0xffffffff81a9d77e in fbt_getargdesc (arg=Variable "arg" is not
 available.
 )
     at /usr/home/rstone/freebsd/head_2/sys/modules/dtrace/fbt/../../../cddl/dev/
 fbt/fbt.c:1264
 #8  0xffffffff8183ac84 in dtrace_ioctl (dev=Variable "dev" is not available.
 ) at dtrace_ioctl.c:613
 #9  0xffffffff807a195a in devfs_ioctl_f (fp=0xfffffe00052aa140,
     com=3238819849, data=Variable "data" is not available.
 )
     at /usr/home/rstone/freebsd/head_2/sys/fs/devfs/devfs_vnops.c:757
 #10 0xffffffff8090514d in kern_ioctl (td=Variable "td" is not available.
 ) at file.h:287
 #11 0xffffffff809053ed in sys_ioctl (td=0xfffffe00054ed900,
     uap=0xffffff8025f57bb0)
     at /usr/home/rstone/freebsd/head_2/sys/kern/sys_generic.c:691
 #12 0xffffffff80bb3f60 in amd64_syscall (td=0xfffffe00054ed900, traced=0)
     at subr_syscall.c:135
 #13 0xffffffff80b9f8b7 in Xfast_syscall ()
     at /usr/home/rstone/freebsd/head_2/sys/amd64/amd64/exception.S:387
 #14 0x0000000801a47e3c in ?? ()

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/166928: commit references a PR
Date: Tue, 13 Aug 2013 03:11:01 +0000 (UTC)

 Author: markj
 Date: Tue Aug 13 03:10:39 2013
 New Revision: 254268
 URL: http://svnweb.freebsd.org/changeset/base/254268
 
 Log:
   FreeBSD's DTrace implementation has a few problems with respect to handling
   probes declared in a kernel module when that module is unloaded. In
   particular,
   
   * Unloading a module with active SDT probes will cause a panic. [1]
   * A module's (FBT/SDT) probes aren't destroyed when the module is unloaded;
     trying to use them after the fact will generally cause a panic.
   
   This change fixes both problems by porting the DTrace module load/unload
   handlers from illumos and registering them with the corresponding
   EVENTHANDLER(9) handlers. This allows the DTrace framework to destroy all
   probes defined in a module when that module is unloaded, and to prevent a
   module unload from proceeding if some of its probes are active. The latter
   problem has already been fixed for FBT probes by checking lf->nenabled in
   kern_kldunload(), but moving the check into the DTrace framework generalizes
   it to all kernel providers and also fixes a race in the current
   implementation (since a probe may be activated between the check and the
   call to linker_file_unload()).
   
   Additionally, the SDT implementation has been reworked to define SDT
   providers/probes/argtypes in linker sets rather than using SYSINIT/SYSUNINIT
   to create and destroy SDT probes when a module is loaded or unloaded. This
   simplifies things quite a bit since it means that pretty much all of the SDT
   code can live in sdt.ko, and since it becomes easier to integrate SDT with
   the DTrace framework. Furthermore, this allows FreeBSD to be quite flexible
   in that SDT providers spanning multiple modules can be created on the fly
   when a module is loaded; at the moment it looks like illumos' SDT
   implementation requires all SDT probes to be statically defined in a single
   kernel table.
   
   PR:		166927, 166926, 166928
   Reported by:	davide [1]
   Reviewed by:	avg, trociny (earlier version)
   MFC after:	1 month
 
 Modified:
   head/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c
   head/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h
   head/sys/cddl/dev/dtrace/dtrace_load.c
   head/sys/cddl/dev/dtrace/dtrace_unload.c
   head/sys/cddl/dev/fbt/fbt.c
   head/sys/cddl/dev/sdt/sdt.c
   head/sys/kern/kern_linker.c
   head/sys/kern/kern_sdt.c
   head/sys/sys/sdt.h
 
 Modified: head/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c
 ==============================================================================
 --- head/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -116,6 +116,7 @@
  #if !defined(sun)
  #include <sys/callout.h>
  #include <sys/ctype.h>
 +#include <sys/eventhandler.h>
  #include <sys/limits.h>
  #include <sys/kdb.h>
  #include <sys/kernel.h>
 @@ -240,6 +241,8 @@ int		dtrace_in_probe;	/* non-zero if exe
  #if defined(__i386__) || defined(__amd64__) || defined(__mips__) || defined(__powerpc__)
  uintptr_t	dtrace_in_probe_addr;	/* Address of invop when already in probe */
  #endif
 +static eventhandler_tag	dtrace_modload_tag;
 +static eventhandler_tag	dtrace_modunload_tag;
  #endif
  
  /*
 @@ -8098,19 +8101,6 @@ dtrace_probe_description(const dtrace_pr
  	(void) strncpy(pdp->dtpd_name, prp->dtpr_name, DTRACE_NAMELEN - 1);
  }
  
 -#if !defined(sun)
 -static int
 -dtrace_probe_provide_cb(linker_file_t lf, void *arg)
 -{
 -	dtrace_provider_t *prv = (dtrace_provider_t *) arg;
 -
 -	prv->dtpv_pops.dtps_provide_module(prv->dtpv_arg, lf);
 -
 -	return(0);
 -}
 -#endif
 -
 -
  /*
   * Called to indicate that a probe -- or probes -- should be provided by a
   * specfied provider.  If the specified description is NULL, the provider will
 @@ -8166,8 +8156,6 @@ dtrace_probe_provide(dtrace_probedesc_t 
  		} while ((ctl = ctl->mod_next) != &modules);
  
  		mutex_exit(&mod_lock);
 -#else
 -		(void) linker_file_foreach(dtrace_probe_provide_cb, prv);
  #endif
  	} while (all && (prv = prv->dtpv_next) != NULL);
  }
 @@ -15152,7 +15140,6 @@ dtrace_helpers_duplicate(proc_t *from, p
  		dtrace_helper_provider_register(to, newhelp, NULL);
  }
  
 -#if defined(sun)
  /*
   * DTrace Hook Functions
   */
 @@ -15166,7 +15153,9 @@ dtrace_module_loaded(modctl_t *ctl)
  	mutex_enter(&mod_lock);
  #endif
  
 +#if defined(sun)
  	ASSERT(ctl->mod_busy);
 +#endif
  
  	/*
  	 * We're going to call each providers per-module provide operation
 @@ -15214,12 +15203,29 @@ dtrace_module_loaded(modctl_t *ctl)
  }
  
  static void
 +#if defined(sun)
  dtrace_module_unloaded(modctl_t *ctl)
 +#else
 +dtrace_module_unloaded(modctl_t *ctl, int *error)
 +#endif
  {
  	dtrace_probe_t template, *probe, *first, *next;
  	dtrace_provider_t *prov;
 +#if !defined(sun)
 +	char modname[DTRACE_MODNAMELEN];
 +	size_t len;
 +#endif
  
 +#if defined(sun)
  	template.dtpr_mod = ctl->mod_modname;
 +#else
 +	/* Handle the fact that ctl->filename may end in ".ko". */
 +	strlcpy(modname, ctl->filename, sizeof(modname));
 +	len = strlen(ctl->filename);
 +	if (len > 3 && strcmp(modname + len - 3, ".ko") == 0)
 +		modname[len - 3] = '\0';
 +	template.dtpr_mod = modname;
 +#endif
  
  	mutex_enter(&dtrace_provider_lock);
  #if defined(sun)
 @@ -15227,6 +15233,18 @@ dtrace_module_unloaded(modctl_t *ctl)
  #endif
  	mutex_enter(&dtrace_lock);
  
 +#if !defined(sun)
 +	if (ctl->nenabled > 0) {
 +		/* Don't allow unloads if a probe is enabled. */
 +		mutex_exit(&dtrace_provider_lock);
 +		mutex_exit(&dtrace_lock);
 +		*error = -1;
 +		printf(
 +	"kldunload: attempt to unload module that has DTrace probes enabled\n");
 +		return;
 +	}
 +#endif
 +
  	if (dtrace_bymod == NULL) {
  		/*
  		 * The DTrace module is loaded (obviously) but not attached;
 @@ -15260,8 +15278,13 @@ dtrace_module_unloaded(modctl_t *ctl)
  			 * probe, either.
  			 */
  			if (dtrace_err_verbose) {
 +#if defined(sun)
  				cmn_err(CE_WARN, "unloaded module '%s' had "
  				    "enabled probes", ctl->mod_modname);
 +#else
 +				cmn_err(CE_WARN, "unloaded module '%s' had "
 +				    "enabled probes", modname);
 +#endif
  			}
  
  			return;
 @@ -15304,7 +15327,11 @@ dtrace_module_unloaded(modctl_t *ctl)
  		kmem_free(probe->dtpr_mod, strlen(probe->dtpr_mod) + 1);
  		kmem_free(probe->dtpr_func, strlen(probe->dtpr_func) + 1);
  		kmem_free(probe->dtpr_name, strlen(probe->dtpr_name) + 1);
 +#if defined(sun)
  		vmem_free(dtrace_arena, (void *)(uintptr_t)probe->dtpr_id, 1);
 +#else
 +		free_unr(dtrace_arena, probe->dtpr_id);
 +#endif
  		kmem_free(probe, sizeof (dtrace_probe_t));
  	}
  
 @@ -15315,6 +15342,26 @@ dtrace_module_unloaded(modctl_t *ctl)
  	mutex_exit(&dtrace_provider_lock);
  }
  
 +#if !defined(sun)
 +static void
 +dtrace_mod_load(void *arg __unused, linker_file_t lf)
 +{
 +
 +	dtrace_module_loaded(lf);
 +}
 +
 +static void
 +dtrace_mod_unload(void *arg __unused, linker_file_t lf, int *error)
 +{
 +
 +	if (*error != 0)
 +		/* We already have an error, so don't do anything. */
 +		return;
 +	dtrace_module_unloaded(lf, error);
 +}
 +#endif
 +
 +#if defined(sun)
  static void
  dtrace_suspend(void)
  {
 
 Modified: head/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h
 ==============================================================================
 --- head/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -2312,8 +2312,10 @@ extern void dtrace_membar_producer(void)
  extern void dtrace_membar_consumer(void);
  
  extern void (*dtrace_cpu_init)(processorid_t);
 +#if defined(sun)
  extern void (*dtrace_modload)(modctl_t *);
  extern void (*dtrace_modunload)(modctl_t *);
 +#endif
  extern void (*dtrace_helpers_cleanup)(void);
  extern void (*dtrace_helpers_fork)(proc_t *parent, proc_t *child);
  extern void (*dtrace_cpustart_init)(void);
 
 Modified: head/sys/cddl/dev/dtrace/dtrace_load.c
 ==============================================================================
 --- head/sys/cddl/dev/dtrace/dtrace_load.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/cddl/dev/dtrace/dtrace_load.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -56,6 +56,12 @@ dtrace_load(void *dummy)
  	/* Hang our hook for exceptions. */
  	dtrace_invop_init();
  
 +	/* Register callbacks for module load and unload events. */
 +	dtrace_modload_tag = EVENTHANDLER_REGISTER(mod_load,
 +	    dtrace_mod_load, NULL, EVENTHANDLER_PRI_ANY);
 +	dtrace_modunload_tag = EVENTHANDLER_REGISTER(mod_unload,
 +	    dtrace_mod_unload, NULL, EVENTHANDLER_PRI_ANY);
 +
  	/*
  	 * Initialise the mutexes without 'witness' because the dtrace
  	 * code is mostly written to wait for memory. To have the
 
 Modified: head/sys/cddl/dev/dtrace/dtrace_unload.c
 ==============================================================================
 --- head/sys/cddl/dev/dtrace/dtrace_unload.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/cddl/dev/dtrace/dtrace_unload.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -67,6 +67,8 @@ dtrace_unload()
  	}
  
  	dtrace_provider = NULL;
 +	EVENTHANDLER_DEREGISTER(mod_load, dtrace_modload_tag);
 +	EVENTHANDLER_DEREGISTER(mod_unload, dtrace_modunload_tag);
  
  	if ((state = dtrace_anon_grab()) != NULL) {
  		/*
 
 Modified: head/sys/cddl/dev/fbt/fbt.c
 ==============================================================================
 --- head/sys/cddl/dev/fbt/fbt.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/cddl/dev/fbt/fbt.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -1335,6 +1335,15 @@ fbt_getargdesc(void *arg __unused, dtrac
  	return;
  }
  
 +static int
 +fbt_linker_file_cb(linker_file_t lf, void *arg)
 +{
 +
 +	fbt_provide_module(arg, lf);
 +
 +	return (0);
 +}
 +
  static void
  fbt_load(void *dummy)
  {
 @@ -1359,8 +1368,10 @@ fbt_load(void *dummy)
  	if (dtrace_register("fbt", &fbt_attr, DTRACE_PRIV_USER,
  	    NULL, &fbt_pops, NULL, &fbt_id) != 0)
  		return;
 -}
  
 +	/* Create probes for the kernel and already-loaded modules. */
 +	linker_file_foreach(fbt_linker_file_cb, NULL);
 +}
  
  static int
  fbt_unload()
 
 Modified: head/sys/cddl/dev/sdt/sdt.c
 ==============================================================================
 --- head/sys/cddl/dev/sdt/sdt.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/cddl/dev/sdt/sdt.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -24,36 +24,44 @@
   *
   */
  
 -#ifndef KDTRACE_HOOKS
 -#define KDTRACE_HOOKS
 -#endif
 +#include "opt_kdtrace.h"
  
  #include <sys/cdefs.h>
  #include <sys/param.h>
  #include <sys/systm.h>
 +
  #include <sys/conf.h>
 +#include <sys/eventhandler.h>
  #include <sys/kernel.h>
  #include <sys/limits.h>
 -#include <sys/lock.h>
  #include <sys/linker.h>
 +#include <sys/linker_set.h>
 +#include <sys/lock.h>
 +#include <sys/malloc.h>
  #include <sys/module.h>
  #include <sys/mutex.h>
 -
 -#include <sys/dtrace.h>
 +#include <sys/queue.h>
  #include <sys/sdt.h>
  
 -#define SDT_ADDR2NDX(addr) (((uintptr_t)(addr)) >> 4)
 +#include <sys/dtrace.h>
 +#include <sys/dtrace_bsd.h>
  
 -static d_open_t sdt_open;
 -static int	sdt_unload(void);
 +/* DTrace methods. */
  static void	sdt_getargdesc(void *, dtrace_id_t, void *, dtrace_argdesc_t *);
  static void	sdt_provide_probes(void *, dtrace_probedesc_t *);
  static void	sdt_destroy(void *, dtrace_id_t, void *);
  static void	sdt_enable(void *, dtrace_id_t, void *);
  static void	sdt_disable(void *, dtrace_id_t, void *);
 +
 +static d_open_t	sdt_open;
  static void	sdt_load(void *);
 -static int	sdt_provider_unreg_callback(struct sdt_provider *prov, 
 -		    void *arg);
 +static int	sdt_unload(void *);
 +static void	sdt_create_provider(struct sdt_provider *);
 +static void	sdt_create_probe(struct sdt_probe *);
 +static void	sdt_modload(void *, struct linker_file *);
 +static void	sdt_modunload(void *, struct linker_file *, int *);
 +
 +static MALLOC_DEFINE(M_SDT, "SDT", "DTrace SDT providers");
  
  static struct cdevsw sdt_cdevsw = {
  	.d_version	= D_VERSION,
 @@ -79,141 +87,261 @@ static dtrace_pops_t sdt_pops = {
  	sdt_getargdesc,
  	NULL,
  	NULL,
 -	sdt_destroy
 +	sdt_destroy,
  };
  
 -static struct cdev		*sdt_cdev;
 +static struct cdev	*sdt_cdev;
  
 -static int
 -sdt_argtype_callback(struct sdt_argtype *argtype, void *arg)
 -{
 -	dtrace_argdesc_t *desc = arg;
 +static TAILQ_HEAD(, sdt_provider) sdt_prov_list;
  
 -	if (desc->dtargd_ndx == argtype->ndx) {
 -		desc->dtargd_mapping = desc->dtargd_ndx; /* XXX */
 -		strlcpy(desc->dtargd_native, argtype->type,
 -		    sizeof(desc->dtargd_native));
 -		desc->dtargd_xlate[0] = '\0'; /* XXX */
 -	}
 -
 -	return (0);
 -}
 +eventhandler_tag	modload_tag;
 +eventhandler_tag	modunload_tag;
  
  static void
 -sdt_getargdesc(void *arg, dtrace_id_t id, void *parg, dtrace_argdesc_t *desc)
 +sdt_create_provider(struct sdt_provider *prov)
  {
 -	struct sdt_probe *probe = parg;
 +	struct sdt_provider *curr, *newprov;
  
 -	if (desc->dtargd_ndx < probe->n_args)
 -		(void) (sdt_argtype_listall(probe, sdt_argtype_callback, desc));
 -	else
 -		desc->dtargd_ndx = DTRACE_ARGNONE;
 +	TAILQ_FOREACH(curr, &sdt_prov_list, prov_entry)
 +		if (strcmp(prov->name, curr->name) == 0) {
 +			/* The provider has already been defined. */
 +			curr->sdt_refs++;
 +			return;
 +		}
  
 -	return;
 +	/*
 +	 * Make a copy of prov so that we don't lose fields if its module is
 +	 * unloaded but the provider isn't destroyed. This could happen with
 +	 * a provider that spans multiple modules.
 +	 */
 +	newprov = malloc(sizeof(*newprov), M_SDT, M_WAITOK | M_ZERO);
 +	newprov->name = strdup(prov->name, M_SDT);
 +	prov->sdt_refs = newprov->sdt_refs = 1;
 +	TAILQ_INIT(&newprov->probe_list);
 +
 +	TAILQ_INSERT_TAIL(&sdt_prov_list, newprov, prov_entry);
 +
 +	(void)dtrace_register(newprov->name, &sdt_attr, DTRACE_PRIV_USER, NULL,
 +	    &sdt_pops, NULL, (dtrace_provider_id_t *)&newprov->id);
 +	prov->id = newprov->id;
  }
  
 -static int
 -sdt_probe_callback(struct sdt_probe *probe, void *arg __unused)
 +static void
 +sdt_create_probe(struct sdt_probe *probe)
  {
 -	struct sdt_provider *prov = probe->prov;
 -	char mod[64];
 -	char func[64];
 -	char name[64];
 +	struct sdt_provider *prov;
 +	char mod[DTRACE_MODNAMELEN];
 +	char func[DTRACE_FUNCNAMELEN];
 +	char name[DTRACE_NAMELEN];
 +	size_t len;
 +
 +	TAILQ_FOREACH(prov, &sdt_prov_list, prov_entry)
 +		if (strcmp(prov->name, probe->prov->name) == 0)
 +			break;
 +
 +	KASSERT(prov != NULL, ("probe defined without a provider"));
 +
 +	/* If no module name was specified, use the module filename. */
 +	if (*probe->mod == 0) {
 +		len = strlcpy(mod, probe->sdtp_lf->filename, sizeof(mod));
 +		if (len > 3 && strcmp(mod + len - 3, ".ko") == 0)
 +			mod[len - 3] = '\0';
 +	} else
 +		strlcpy(mod, probe->mod, sizeof(mod));
  
  	/*
  	 * Unfortunately this is necessary because the Solaris DTrace
  	 * code mixes consts and non-consts with casts to override
  	 * the incompatibilies. On FreeBSD, we use strict warnings
 -	 * in gcc, so we have to respect const vs non-const.
 +	 * in the C compiler, so we have to respect const vs non-const.
  	 */
 -	strlcpy(mod, probe->mod, sizeof(mod));
  	strlcpy(func, probe->func, sizeof(func));
  	strlcpy(name, probe->name, sizeof(name));
  
 -	if (dtrace_probe_lookup(prov->id, mod, func, name) != 0)
 -		return (0);
 +	if (dtrace_probe_lookup(prov->id, mod, func, name) != DTRACE_IDNONE)
 +		return;
  
 -	(void) dtrace_probe_create(prov->id, probe->mod, probe->func,
 -	    probe->name, 1, probe);
 +	TAILQ_INSERT_TAIL(&prov->probe_list, probe, probe_entry);
  
 -	return (0);
 +	(void)dtrace_probe_create(prov->id, mod, func, name, 1, probe);
  }
  
 -static int
 -sdt_provider_entry(struct sdt_provider *prov, void *arg)
 +/* Probes are created through the SDT module load/unload hook. */
 +static void
 +sdt_provide_probes(void *arg, dtrace_probedesc_t *desc)
  {
 -	return (sdt_probe_listall(prov, sdt_probe_callback, NULL));
  }
  
  static void
 -sdt_provide_probes(void *arg, dtrace_probedesc_t *desc)
 +sdt_enable(void *arg __unused, dtrace_id_t id, void *parg)
  {
 -	if (desc != NULL)
 -		return;
 +	struct sdt_probe *probe = parg;
  
 -	(void) sdt_provider_listall(sdt_provider_entry, NULL);
 +	probe->id = id;
 +	probe->sdtp_lf->nenabled++;
  }
  
  static void
 -sdt_destroy(void *arg, dtrace_id_t id, void *parg)
 +sdt_disable(void *arg __unused, dtrace_id_t id, void *parg)
  {
 -	/* Nothing to do here. */
 +	struct sdt_probe *probe = parg;
 +
 +	KASSERT(probe->sdtp_lf->nenabled > 0, ("no probes enabled"));
 +
 +	probe->id = 0;
 +	probe->sdtp_lf->nenabled--;
  }
  
  static void
 -sdt_enable(void *arg, dtrace_id_t id, void *parg)
 +sdt_getargdesc(void *arg, dtrace_id_t id, void *parg, dtrace_argdesc_t *desc)
  {
 +	struct sdt_argtype *argtype;
  	struct sdt_probe *probe = parg;
  
 -	probe->id = id;
 +	if (desc->dtargd_ndx < probe->n_args) {
 +		TAILQ_FOREACH(argtype, &probe->argtype_list, argtype_entry) {
 +			if (desc->dtargd_ndx == argtype->ndx) {
 +				/* XXX */
 +				desc->dtargd_mapping = desc->dtargd_ndx;
 +				strlcpy(desc->dtargd_native, argtype->type,
 +				    sizeof(desc->dtargd_native));
 +				desc->dtargd_xlate[0] = '\0'; /* XXX */
 +			}
 +		}
 +	} else
 +		desc->dtargd_ndx = DTRACE_ARGNONE;
  }
  
  static void
 -sdt_disable(void *arg, dtrace_id_t id, void *parg)
 +sdt_destroy(void *arg, dtrace_id_t id, void *parg)
  {
 -	struct sdt_probe *probe = parg;
 +	struct sdt_probe *probe;
  
 -	probe->id = 0;
 +	probe = parg;
 +	TAILQ_REMOVE(&probe->prov->probe_list, probe, probe_entry);
 +}
 +
 +/*
 + * Called from the kernel linker when a module is loaded, before
 + * dtrace_module_loaded() is called. This is done so that it's possible to
 + * register new providers when modules are loaded. We cannot do this in the
 + * provide_module method since it's called with the provider lock held
 + * and dtrace_register() will try to acquire it again.
 + */
 +static void
 +sdt_modload(void *arg __unused, struct linker_file *lf)
 +{
 +	struct sdt_provider **prov, **begin, **end;
 +	struct sdt_probe **probe, **p_begin, **p_end;
 +	struct sdt_argtype **argtype, **a_begin, **a_end;
 +
 +	if (linker_file_lookup_set(lf, "sdt_providers_set", &begin, &end, NULL))
 +		return;
 +	for (prov = begin; prov < end; prov++)
 +		sdt_create_provider(*prov);
 +
 +	if (linker_file_lookup_set(lf, "sdt_probes_set", &p_begin, &p_end,
 +	    NULL))
 +		return;
 +	for (probe = p_begin; probe < p_end; probe++) {
 +		(*probe)->sdtp_lf = lf;
 +		sdt_create_probe(*probe);
 +		TAILQ_INIT(&(*probe)->argtype_list);
 +	}
 +
 +	if (linker_file_lookup_set(lf, "sdt_argtypes_set", &a_begin, &a_end,
 +	    NULL))
 +		return;
 +	for (argtype = a_begin; argtype < a_end; argtype++) {
 +		(*argtype)->probe->n_args++;
 +		TAILQ_INSERT_TAIL(&(*argtype)->probe->argtype_list, *argtype,
 +		    argtype_entry);
 +	}
 +}
 +
 +static void
 +sdt_modunload(void *arg __unused, struct linker_file *lf, int *error __unused)
 +{
 +	struct sdt_provider *prov, **curr, **begin, **end, *tmp;
 +
 +	if (*error != 0)
 +		/* We already have an error, so don't do anything. */
 +		return;
 +	else if (linker_file_lookup_set(lf, "sdt_providers_set", &begin, &end, NULL))
 +		/* No DTrace providers are declared in this module. */
 +		return;
 +
 +	/*
 +	 * Go through all the providers declared in this module and unregister
 +	 * any that aren't declared in another loaded module.
 +	 */
 +	for (curr = begin; curr < end; curr++) {
 +		TAILQ_FOREACH_SAFE(prov, &sdt_prov_list, prov_entry, tmp) {
 +			if (strcmp(prov->name, (*curr)->name) == 0) {
 +				if (prov->sdt_refs == 1) {
 +					TAILQ_REMOVE(&sdt_prov_list, prov,
 +					    prov_entry);
 +					dtrace_unregister(prov->id);
 +					free(prov->name, M_SDT);
 +					free(prov, M_SDT);
 +				} else
 +					prov->sdt_refs--;
 +				break;
 +			}
 +		}
 +	}
  }
  
  static int
 -sdt_provider_reg_callback(struct sdt_provider *prov, void *arg __unused)
 +sdt_linker_file_cb(linker_file_t lf, void *arg __unused)
  {
 -	return (dtrace_register(prov->name, &sdt_attr, DTRACE_PRIV_USER,
 -	    NULL, &sdt_pops, NULL, (dtrace_provider_id_t *) &prov->id));
 +
 +	sdt_modload(NULL, lf);
 +
 +	return (0);
  }
  
  static void
 -sdt_load(void *dummy)
 +sdt_load(void *arg __unused)
  {
 +
 +	TAILQ_INIT(&sdt_prov_list);
 +
  	/* Create the /dev/dtrace/sdt entry. */
  	sdt_cdev = make_dev(&sdt_cdevsw, 0, UID_ROOT, GID_WHEEL, 0600,
  	    "dtrace/sdt");
  
  	sdt_probe_func = dtrace_probe;
  
 -	sdt_register_callbacks(sdt_provider_reg_callback, NULL,
 -	    sdt_provider_unreg_callback, NULL, sdt_probe_callback, NULL);
 -}
 +	modload_tag = EVENTHANDLER_REGISTER(mod_load, sdt_modload, NULL,
 +	    EVENTHANDLER_PRI_ANY);
 +	modunload_tag = EVENTHANDLER_REGISTER(mod_unload, sdt_modunload, NULL,
 +	    EVENTHANDLER_PRI_ANY);
  
 -static int
 -sdt_provider_unreg_callback(struct sdt_provider *prov, void *arg __unused)
 -{
 -	return (dtrace_unregister(prov->id));
 +	/* Pick up probes from the kernel and already-loaded modules. */
 +	linker_file_foreach(sdt_linker_file_cb, NULL);
  }
  
  static int
 -sdt_unload()
 +sdt_unload(void *arg __unused)
  {
 -	int error = 0;
 +	struct sdt_provider *prov, *tmp;
 +
 +	EVENTHANDLER_DEREGISTER(mod_load, modload_tag);
 +	EVENTHANDLER_DEREGISTER(mod_unload, modunload_tag);
  
  	sdt_probe_func = sdt_probe_stub;
  
 -	sdt_deregister_callbacks();
 -	
 +	TAILQ_FOREACH_SAFE(prov, &sdt_prov_list, prov_entry, tmp) {
 +		TAILQ_REMOVE(&sdt_prov_list, prov, prov_entry);
 +		dtrace_unregister(prov->id);
 +		free(prov->name, M_SDT);
 +		free(prov, M_SDT);
 +	}
 +
  	destroy_dev(sdt_cdev);
  
 -	return (error);
 +	return (0);
  }
  
  /* ARGSUSED */
 @@ -235,7 +363,6 @@ sdt_modevent(module_t mod __unused, int 
  	default:
  		error = EOPNOTSUPP;
  		break;
 -
  	}
  
  	return (error);
 @@ -243,8 +370,10 @@ sdt_modevent(module_t mod __unused, int 
  
  /* ARGSUSED */
  static int
 -sdt_open(struct cdev *dev __unused, int oflags __unused, int devtype __unused, struct thread *td __unused)
 +sdt_open(struct cdev *dev __unused, int oflags __unused, int devtype __unused,
 +    struct thread *td __unused)
  {
 +
  	return (0);
  }
  
 
 Modified: head/sys/kern/kern_linker.c
 ==============================================================================
 --- head/sys/kern/kern_linker.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/kern/kern_linker.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -1106,11 +1106,7 @@ kern_kldunload(struct thread *td, int fi
  		EVENTHANDLER_INVOKE(mod_unload, lf, &error);
  		if (error != 0)
  			error = EBUSY;
 -		else if (lf->nenabled > 0) {
 -			printf("kldunload: attempt to unload file that has"
 -			    " DTrace probes enabled\n");
 -			error = EBUSY;
 -		} else if (lf->userrefs == 0) {
 +		else if (lf->userrefs == 0) {
  			/*
  			 * XXX: maybe LINKER_UNLOAD_FORCE should override ?
  			 */
 
 Modified: head/sys/kern/kern_sdt.c
 ==============================================================================
 --- head/sys/kern/kern_sdt.c	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/kern/kern_sdt.c	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -23,317 +23,29 @@
   * SUCH DAMAGE.
   *
   * $FreeBSD$
 - *
 - * Backend for the Statically Defined Tracing (SDT) kernel support. This is
 - * required to allow a module to load even though DTrace kernel support may
 - * not be present. A module may be built with SDT probes in it which are
 - * registered and deregistered via SYSINIT/SYSUNINIT.
 - *
   */
  
  #include "opt_kdtrace.h"
  
 -#include <sys/cdefs.h>
  #include <sys/param.h>
  #include <sys/systm.h>
 -#include <sys/kernel.h>
 -#include <sys/linker.h>
 -#include <sys/lock.h>
 -#include <sys/proc.h>
 -#include <sys/sx.h>
  #include <sys/sdt.h>
  
  /*
 - * This is the list of statically defined tracing providers.
 - */
 -static TAILQ_HEAD(sdt_provider_list_head, sdt_provider) sdt_provider_list;
 -
 -/*
 - * Mutex to serialise access to the SDT provider list.
 - */
 -static struct sx sdt_sx;
 -
 -/*
 - * Hook for the DTrace probe function. The 'sdt' provider will set this
 - * to dtrace_probe when it loads.
 + * Hook for the DTrace probe function. The SDT provider will set this to
 + * dtrace_probe() when it loads.
   */
  sdt_probe_func_t sdt_probe_func = sdt_probe_stub;
  
 -static sdt_provider_listall_func_t sdt_provider_register_func = NULL;
 -static sdt_provider_listall_func_t sdt_provider_deregister_func = NULL;
 -static sdt_probe_listall_func_t sdt_probe_register_func = NULL;
 -
 -static void *sdt_provider_register_arg;
 -static void *sdt_provider_deregister_arg;
 -static void *sdt_probe_register_arg;
 -
 -static int sdt_provider_listall_locked(sdt_provider_listall_func_t, void *);
 -
  /*
   * This is a stub for probe calls in case kernel DTrace support isn't
 - * compiled in. It should never get called because there is no DTrace
 - * support to enable it.
 + * enabled. It should never get called because there is no DTrace support
 + * to enable it.
   */
  void
  sdt_probe_stub(uint32_t id, uintptr_t arg0, uintptr_t arg1,
      uintptr_t arg2, uintptr_t arg3, uintptr_t arg4)
  {
 -	printf("sdt_probe_stub: Why did this get called?\n");
 -}
 -
 -/*
 - * Called from SYSINIT to register a provider.
 - */
 -void
 -sdt_provider_register(void *arg)
 -{
 -	struct sdt_provider *prov = arg;
 -
 -	sx_xlock(&sdt_sx);
 -
 -	TAILQ_INSERT_TAIL(&sdt_provider_list, prov, prov_entry);
 -
 -	TAILQ_INIT(&prov->probe_list);
 -
 -	if (sdt_provider_register_func != NULL)
 -		sdt_provider_register_func(prov, sdt_provider_register_arg);
 -
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -/*
 - * Called from SYSUNINIT to de-register a provider.
 - */
 -void
 -sdt_provider_deregister(void *arg)
 -{
 -	struct sdt_provider *prov = arg;
 -
 -	sx_xlock(&sdt_sx);
 -
 -	TAILQ_REMOVE(&sdt_provider_list, prov, prov_entry);
 -
 -	if (sdt_provider_deregister_func != NULL)
 -		sdt_provider_deregister_func(prov, sdt_provider_deregister_arg);
 -
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -/*
 - * Called from SYSINIT to register a statically defined trace probe.
 - */
 -void
 -sdt_probe_register(void *arg)
 -{
 -	struct sdt_probe *probe = arg;
 -
 -	/*
 -	 * Check the reference structure version. Only version 1 is
 -	 * supported at the moment.
 -	 */
 -	if (probe->version != sizeof(struct sdt_probe)) {
 -		printf("%s:%s:%s has version %d when %d required\n", probe->mod, probe->func, probe->name, probe->version, (int) sizeof(struct sdt_probe));
 -		return;
 -	}
 -
 -	sx_xlock(&sdt_sx);
 -
 -	TAILQ_INSERT_TAIL(&probe->prov->probe_list, probe, probe_entry);
 -
 -	TAILQ_INIT(&probe->argtype_list);
 -
 -	probe->state = SDT_INIT;
 -
 -	if (sdt_probe_register_func != NULL)
 -		sdt_probe_register_func(probe, sdt_provider_register_arg);
 -
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -/*
 - * Called from SYSUNINIT to de-register a statically defined trace probe.
 - */
 -void
 -sdt_probe_deregister(void *arg)
 -{
 -	struct sdt_probe *probe = arg;
 -
 -	sx_xlock(&sdt_sx);
 -
 -	if (probe->state == SDT_INIT) {
 -		TAILQ_REMOVE(&probe->prov->probe_list, probe, probe_entry);
 -		probe->state = SDT_UNINIT;
 -	}
 -
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -/*
 - * Called from SYSINIT to register a statically defined trace probe argument.
 - */
 -void
 -sdt_argtype_register(void *arg)
 -{
 -	struct sdt_argtype *argtype = arg;
 -
 -	sx_xlock(&sdt_sx);
 -
 -	TAILQ_INSERT_TAIL(&argtype->probe->argtype_list, argtype, argtype_entry);
 -
 -	argtype->probe->n_args++;
 -
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -/*
 - * Called from SYSUNINIT to de-register a statically defined trace probe argument.
 - */
 -void
 -sdt_argtype_deregister(void *arg)
 -{
 -	struct sdt_argtype *argtype = arg;
 -
 -	sx_xlock(&sdt_sx);
 -
 -	TAILQ_REMOVE(&argtype->probe->argtype_list, argtype, argtype_entry);
 -
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -static void
 -sdt_init(void *arg)
 -{ 
 -	sx_init_flags(&sdt_sx, "Statically Defined Tracing", SX_NOWITNESS);
 -
 -	TAILQ_INIT(&sdt_provider_list);
 -}
 -
 -SYSINIT(sdt, SI_SUB_KDTRACE, SI_ORDER_FIRST, sdt_init, NULL);
 -
 -static void
 -sdt_uninit(void *arg)
 -{ 
 -	sx_destroy(&sdt_sx);
 -}
 -
 -SYSUNINIT(sdt, SI_SUB_KDTRACE, SI_ORDER_FIRST, sdt_uninit, NULL);
 -
 -/*
 - * List statically defined tracing providers.
 - */
 -int
 -sdt_provider_listall(sdt_provider_listall_func_t callback_func, void *arg)
 -{
 -	int error;
 -
 -	sx_xlock(&sdt_sx);
 -	error = sdt_provider_listall_locked(callback_func, arg);
 -	sx_xunlock(&sdt_sx);
 -
 -	return (error);
 -}
 -
 -static int
 -sdt_provider_listall_locked(sdt_provider_listall_func_t callback_func,
 -    void *arg)
 -{
 -	int error = 0;
 -	struct sdt_provider *prov;
 -
 -	sx_assert(&sdt_sx, SX_XLOCKED);
 -
 -	TAILQ_FOREACH(prov, &sdt_provider_list, prov_entry) {
 -		if ((error = callback_func(prov, arg)) != 0)
 -			break;
 -	}
 -
 -	return (error);
 -}
 -
 -/*
 - * List statically defined tracing probes.
 - */
 -int
 -sdt_probe_listall(struct sdt_provider *prov, 
 -    sdt_probe_listall_func_t callback_func,void *arg)
 -{
 -	int error = 0;
 -	int locked;
 -	struct sdt_probe *probe;
 -
 -	locked = sx_xlocked(&sdt_sx);
 -	if (!locked)
 -		sx_xlock(&sdt_sx);
 -
 -	TAILQ_FOREACH(probe, &prov->probe_list, probe_entry) {
 -		if ((error = callback_func(probe, arg)) != 0)
 -			break;
 -	}
 -
 -	if (!locked)
 -		sx_xunlock(&sdt_sx);
 -
 -	return (error);
 -}
 -
 -/*
 - * List statically defined tracing probe arguments.
 - */
 -int
 -sdt_argtype_listall(struct sdt_probe *probe, 
 -    sdt_argtype_listall_func_t callback_func,void *arg)
 -{
 -	int error = 0;
 -	int locked;
 -	struct sdt_argtype *argtype;
 -
 -	locked = sx_xlocked(&sdt_sx);
 -	if (!locked)
 -		sx_xlock(&sdt_sx);
  
 -	TAILQ_FOREACH(argtype, &probe->argtype_list, argtype_entry) {
 -		if ((error = callback_func(argtype, arg)) != 0)
 -			break;
 -	}
 -
 -	if (!locked)
 -		sx_xunlock(&sdt_sx);
 -
 -	return (error);
 -}
 -
 -void sdt_register_callbacks(sdt_provider_listall_func_t register_prov, 
 -    void *reg_prov_arg, sdt_provider_listall_func_t deregister_prov, 
 -    void *dereg_prov_arg, sdt_probe_listall_func_t register_probe, 
 -    void * reg_probe_arg)
 -{
 -
 -	sx_xlock(&sdt_sx);
 -	sdt_provider_register_func = register_prov;
 -	sdt_provider_deregister_func = deregister_prov;
 -	sdt_probe_register_func = register_probe;
 -
 -	sdt_provider_register_arg = reg_prov_arg;
 -	sdt_provider_deregister_arg = dereg_prov_arg;
 -	sdt_probe_register_arg = reg_probe_arg;
 -
 -	sdt_provider_listall_locked(register_prov, reg_prov_arg);
 -	sx_xunlock(&sdt_sx);
 -}
 -
 -void sdt_deregister_callbacks(void)
 -{
 -
 -	sx_xlock(&sdt_sx);
 -	sdt_provider_listall_locked(sdt_provider_deregister_func, 
 -	    sdt_provider_deregister_arg);
 -
 -	sdt_provider_register_func = NULL;
 -	sdt_provider_deregister_func = NULL;
 -	sdt_probe_register_func = NULL;
 -
 -	sdt_provider_register_arg = NULL;
 -	sdt_provider_deregister_arg = NULL;
 -	sdt_probe_register_arg = NULL;
 -	sx_xunlock(&sdt_sx);
 +	printf("sdt_probe_stub: Why did this get called?\n");
  }
 
 Modified: head/sys/sys/sdt.h
 ==============================================================================
 --- head/sys/sys/sdt.h	Tue Aug 13 03:09:00 2013	(r254267)
 +++ head/sys/sys/sdt.h	Tue Aug 13 03:10:39 2013	(r254268)
 @@ -77,6 +77,9 @@
  
  #else /* _KERNEL */
  
 +#include <sys/cdefs.h>
 +#include <sys/linker_set.h>
 +
  #ifndef KDTRACE_HOOKS
  
  #define SDT_PROVIDER_DEFINE(prov)
 @@ -109,85 +112,26 @@
  
  #else
  
 -/*
 - * This type definition must match that of dtrace_probe. It is defined this
 
 *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: feedback->patched 
State-Changed-By: markj 
State-Changed-When: Tue Sep 3 23:55:40 UTC 2013 
State-Changed-Why:  
This is addressed by r254268. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166928 
Responsible-Changed-From-To: gnn->markj 
Responsible-Changed-By: markj 
Responsible-Changed-When: Sun Dec 29 17:26:46 UTC 2013 
Responsible-Changed-Why:  
I fixed this bug. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166928 
State-Changed-From-To: patched->closed 
State-Changed-By: markj 
State-Changed-When: Sun Dec 29 17:50:59 UTC 2013 
State-Changed-Why:  
r254268 and related changes have been merged to stable/9. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=166928 
>Unformatted:
