From nobody@FreeBSD.org  Thu Dec  9 00:32:48 2010
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4D2E810656A3
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  9 Dec 2010 00:32:48 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (unknown [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 3B5788FC15
	for <freebsd-gnats-submit@FreeBSD.org>; Thu,  9 Dec 2010 00:32:48 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id oB90Wm66064147
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 9 Dec 2010 00:32:48 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id oB90WmYo064146;
	Thu, 9 Dec 2010 00:32:48 GMT
	(envelope-from nobody)
Message-Id: <201012090032.oB90WmYo064146@red.freebsd.org>
Date: Thu, 9 Dec 2010 00:32:48 GMT
From: Ivan Brawley <ivan@brawley.id.au>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Add ZFS ARC statistics into net-mgmt/collectd port
X-Send-Pr-Version: www-3.1
X-GNATS-Notify: ports@bsdserwis.com

>Number:         152947
>Category:       ports
>Synopsis:       Add ZFS ARC statistics into net-mgmt/collectd port
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    miwi
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 09 00:40:09 UTC 2010
>Closed-Date:    Fri Mar 04 09:41:23 UTC 2011
>Last-Modified:  Fri Mar 04 09:41:23 UTC 2011
>Originator:     Ivan Brawley
>Release:        8.1STABLE
>Organization:
>Environment:
FreeBSD bang 8.1-STABLE FreeBSD 8.1-STABLE #3: Thu Nov  4 18:21:39 CST 2010     root@bang:/usr/obj/usr/src/sys/BANG  amd64

>Description:
The net-mgmt/collectd port has code for Solaris to gather ZFS ARC statistics.

However, it uses Solaris' get_kstat_value(3) library call to gather the stats and the configure run only tests for Solaris to gather ZFS ARC stats.

Included is a patch to the port tree net-mgmt/collectd to make the zfs_arc module for FreeBSD. It tests to make sure that at least version 8.0 of FreeBSD is used.

Because changes are needed to the included configure script, I've added the patches to the autoconf files so that a proper configure script can be generated.

I've given a copy of the patches to the zfs_arc.c source and to the autoconf file to the author of the zfs_arc module to work through getting that submitted as part of collectd's code.

Couple of months ago, I did send a copy of this to the port's maintainer and he suggested that I make a PR for it, so here it is (updated to work on port as of 3-Dec-2010).
>How-To-Repeat:

>Fix:


Patch attached with submission follows:

--- net-mgmt/collectd/Makefile.orig	2010-12-07 10:02:42.958651026 +1030
+++ net-mgmt/collectd/Makefile	2010-12-09 10:40:18.041439748 +1030
@@ -13,15 +13,19 @@
 
 MAINTAINER=	ports@bsdserwis.com
 COMMENT=	Systems & network statistics collection daemon
 
 USE_GMAKE=	yes
 GNU_CONFIGURE=	yes
-USE_AUTOTOOLS=	libltdl:22
+USE_AUTOTOOLS=	autoheader:268 aclocal:111 libtool:22 libtoolize:22 automake:19 autoconf:268 libltdl:22
 WANT_GNOME=	yes
 
+AUTOMAKE_ARGS=	--add-missing --copy
+LIBTOOLIZE_ARGS= --ltdl --copy --force
+ACLOCAL_ARGS=
+
 OPTIONS=	CGI		"Install collection.cgi (requires RRDTOOL)" 	Off \
 		BIND		"Enable BIND 9.5+ statistics"			On  \
 		DEBUG		"Enable debugging" 				Off \
 		APACHE		"Input: Apache mod_status (libcurl)" 		Off \
 		APCUPS		"Input: APC UPS (apcupsd)" 			Off \
 		CURL		"Input: CURL generic web statistics" 		Off \
@@ -33,14 +37,13 @@
 		NGINX		"Input: Nginx" 					Off \
 		OPENVPN		"Input: OpenVPN statistics"			Off \
 		PDNS		"Input: PowerDNS" 				Off \
 		PGSQL		"Input: PostgreSQL" 				Off \
 		PING		"Input: Network latency (liboping)" 		On  \
 		SNMP		"Input: SNMP" 					On  \
-		XMMS		"Input: XMMS" 					Off \
-		RRDTOOL		"Output: RRDTool" 				On
+		XMMS		"Input: XMMS" 					Off
 
 MAN1=		collectd.1 collectd-nagios.1 collectdmon.1
 MAN5=		collectd.conf.5 collectd-email.5 collectd-exec.5 \
 		collectd-snmp.5 collectd-unixsock.5 collectd-perl.5 \
 		collectd-java.5 collectd-python.5 types.db.5
 USE_RC_SUBR=	collectd collectdmon
@@ -49,12 +52,20 @@
 
 CONFIGURE_ENV=	CPPFLAGS="-I${LOCALBASE}/include" \
 		LDFLAGS="-L${LOCALBASE}/lib"
 
 .include <bsd.port.pre.mk>
 
+.if ( ${OSVERSION} >= 800000 )
+OPTIONS+=	ZFS_ARC		"Input: ZFS ARC"				Off
+.else
+.undef WITH_ZFS_ARC
+.endif
+
+OPTIONS+=	RRDTOOL		"Output: RRDTool" 				On
+
 .if ( ${OSVERSION} < 601103 )
 BROKEN=		Need bind9 import post 6.1
 .endif
 
 .if ${OSVERSION} < 700000
 BROKEN=		does not configure on 6.X
@@ -120,13 +131,12 @@
 		--disable-tokyotyrant \
 		--disable-users \
 		--disable-vmem \
 		--disable-vserver \
 		--disable-wireless \
 		--disable-write_http \
-		--disable-zfs_arc \
 		--without-perl-bindings
 
 .if defined(WITH_DEBUG)
 CONFIGURE_ARGS+=--enable-debug
 .endif
 
@@ -304,12 +314,20 @@
 PLIST_SUB+=	XMMS=""
 .else
 CONFIGURE_ARGS+=--disable-xmms
 PLIST_SUB+=	XMMS="@comment "
 .endif
 
+.if defined(WITH_ZFS_ARC) && ( ${OSVERSION} >= 800000 )
+CONFIGURE_ARGS+=--enable-zfs_arc
+PLIST_SUB+=	ZFS_ARC=""
+.else
+CONFIGURE_ARGS+=--disable-zfs_arc
+PLIST_SUB+=	ZFS_ARC="@comment "
+.endif
+
 post-patch:
 	@${REINPLACE_CMD} \
 		-e 's;@prefix@/var/;/var/;' \
 		-e 's;/var/lib/;/var/db/;' \
 		${WRKSRC}/src/collectd.conf.in
 	@${REINPLACE_CMD} \
--- net-mgmt/collectd/files/patch-configure.in.orig	1970-01-01 09:30:00.000000000 +0930
+++ net-mgmt/collectd/files/patch-configure.in	2010-12-08 16:00:08.697721236 +1030
@@ -0,0 +1,59 @@
+--- configure.in.idborig	2010-12-08 15:55:15.333847856 +1030
++++ configure.in	2010-12-08 15:59:28.307013465 +1030
+@@ -63,6 +63,9 @@
+ 	*openbsd*)
+ 	ac_system="OpenBSD"
+ 	;;
++	*freebsd*)
++	ac_system="FreeBSD"
++	;;
+ 	*aix*)
+ 	AC_DEFINE([KERNEL_AIX], 1, [True if program is to be compiled for a AIX kernel])
+ 	ac_system="AIX"
+@@ -90,7 +93,7 @@
+ fi
+ 
+ # Where to install .pc files.
+-pkgconfigdir="${libdir}/pkgconfig"
++pkgconfigdir="${prefix}/libdata/pkgconfig"
+ AC_SUBST(pkgconfigdir)
+ 
+ # Check for standards compliance mode
+@@ -284,6 +287,24 @@
+ 	AC_MSG_RESULT([no])
+ fi
+ 
++AC_MSG_CHECKING([for sysctl kstat.zfs])
++if test -x /sbin/sysctl
++then
++	/sbin/sysctl kstat.zfs 2>/dev/null
++	if test $? -eq 0
++	then
++		AC_MSG_RESULT([yes])
++		AC_DEFINE(HAVE_SYSCTL_KSTAT_ZFS, 1,
++		[Define if sysctl supports kstat.zfs])
++		have_sysctl_kstat_zfs="yes"
++	else
++		AC_MSG_RESULT([no])
++		have_sysctl_kstat_zfs="no"
++	fi
++else
++	AC_MSG_RESULT([no])
++fi
++
+ # For hddtemp module
+ AC_CHECK_HEADERS(linux/major.h libgen.h)
+ 
+@@ -3968,6 +3989,12 @@
+ 	plugin_zfs_arc="yes"
+ fi
+ 
++# FreeBSD with zfs.ko loaded
++if test "x$ac_system" = "xFreeBSD" -a "X$have_sysctl_kstat_zfs" = "Xyes"
++then
++	plugin_zfs_arc="yes"
++fi
++
+ if test "x$with_devinfo$with_kstat" = "xyesyes"
+ then
+ 	plugin_cpu="yes"
--- net-mgmt/collectd/files/patch-src__Makefile.am.orig	1970-01-01 09:30:00.000000000 +0930
+++ net-mgmt/collectd/files/patch-src__Makefile.am	2010-12-08 16:03:52.520952306 +1030
@@ -0,0 +1,24 @@
+--- src/Makefile.am.idborig	2010-12-08 16:00:22.720212550 +1030
++++ src/Makefile.am	2010-12-08 16:02:58.038149584 +1030
+@@ -1158,7 +1158,9 @@
+ zfs_arc_la_SOURCES = zfs_arc.c
+ zfs_arc_la_CFLAGS = $(AM_CFLAGS)
+ zfs_arc_la_LDFLAGS = -module -avoid-version
++if BUILD_WITH_LIBKSTAT
+ zfs_arc_la_LIBADD = -lkstat
++endif
+ collectd_LDADD += "-dlopen" zfs_arc.la
+ collectd_DEPENDENCIES += zfs_arc.la
+ endif
+@@ -1215,9 +1217,9 @@
+ 	$(mkinstalldirs) $(DESTDIR)$(sysconfdir)
+ 	if test -e $(DESTDIR)$(sysconfdir)/collectd.conf; \
+ 	then \
+-		$(INSTALL) -m 0640 collectd.conf $(DESTDIR)$(sysconfdir)/collectd.conf.pkg-orig; \
++		 $(INSTALL) -m 0640 collectd.conf $(DESTDIR)$(sysconfdir)/collectd.conf.sample.pkg-orig; \
+ 	else \
+-		$(INSTALL) -m 0640 collectd.conf $(DESTDIR)$(sysconfdir)/collectd.conf; \
++		$(INSTALL) -m 0640 collectd.conf $(DESTDIR)$(sysconfdir)/collectd.conf.sample; \
+ 	fi; \
+ 	$(mkinstalldirs) $(DESTDIR)$(pkgdatadir)
+ 	$(INSTALL) -m 0644 $(srcdir)/types.db $(DESTDIR)$(pkgdatadir)/types.db;
--- net-mgmt/collectd/files/patch-src__zfs_arc.c.orig	1970-01-01 09:30:00.000000000 +0930
+++ net-mgmt/collectd/files/patch-src__zfs_arc.c	2010-12-08 16:05:55.483845824 +1030
@@ -0,0 +1,100 @@
+--- src/zfs_arc.c.idborig	2010-12-08 16:04:08.946558865 +1030
++++ src/zfs_arc.c	2010-12-08 16:05:23.979254992 +1030
+@@ -26,8 +26,12 @@
+ /*
+  * Global variables
+  */
++#if HAVE_LIBKSTAT
+ static kstat_t *ksp;
+ extern kstat_ctl_t *kc;
++#elif HAVE_SYSCTL
++#include <sys/sysctl.h>
++#endif
+ 
+ static void za_submit (const char* type, const char* type_instance, value_t* values, int values_len)
+ {
+@@ -88,6 +92,19 @@
+ 	za_submit ("arc_counts", type_instance, values, STATIC_ARRAY_SIZE(values));
+ }
+ 
++#if HAVE_SYSCTL
++static uint64_t za_sysctl_read_stat (const char* stat)
++{
++	uint64_t sysctl_stat;
++	size_t stat_size = sizeof(sysctl_stat);
++
++	if (sysctlbyname(stat, &sysctl_stat, &stat_size, NULL, 0) == 0) {
++		return (sysctl_stat);
++	}
++	return(0);
++}
++#endif
++
+ static int za_read (void)
+ {
+ 	gauge_t   arcsize, targetsize, minlimit, maxlimit, hits, misses, l2_size, l2_hits, l2_misses;
+@@ -95,6 +112,7 @@
+ 	counter_t demand_data_misses, demand_metadata_misses, prefetch_data_misses, prefetch_metadata_misses;
+ 	counter_t l2_read_bytes, l2_write_bytes;
+ 
++#if HAVE_LIBKSTAT
+ 	get_kstat (&ksp, "zfs", 0, "arcstats");
+ 	if (ksp == NULL)
+ 	{
+@@ -125,6 +143,31 @@
+ 	l2_write_bytes = get_kstat_value(ksp, "l2_write_bytes");
+ 	l2_hits        = get_kstat_value(ksp, "l2_hits");
+ 	l2_misses      = get_kstat_value(ksp, "l2_misses");
++#elif HAVE_SYSCTL
++	arcsize    = za_sysctl_read_stat("kstat.zfs.misc.arcstats.size");
++	targetsize = za_sysctl_read_stat("kstat.zfs.misc.arcstats.c");
++	minlimit   = za_sysctl_read_stat("kstat.zfs.misc.arcstats.c_min");
++	maxlimit   = za_sysctl_read_stat("kstat.zfs.misc.arcstats.c_max");
++
++	demand_data_hits       = za_sysctl_read_stat("kstat.zfs.misc.arcstats.demand_data_hits");
++	demand_metadata_hits   = za_sysctl_read_stat("kstat.zfs.misc.arcstats.demand_metadata_hits");
++	prefetch_data_hits     = za_sysctl_read_stat("kstat.zfs.misc.arcstats.prefetch_data_hits");
++	prefetch_metadata_hits = za_sysctl_read_stat("kstat.zfs.misc.arcstats.prefetch_metadata_hits");
++
++	demand_data_misses       = za_sysctl_read_stat("kstat.zfs.misc.arcstats.demand_data_misses");
++	demand_metadata_misses   = za_sysctl_read_stat("kstat.zfs.misc.arcstats.demand_metadata_misses");
++	prefetch_data_misses     = za_sysctl_read_stat("kstat.zfs.misc.arcstats.prefetch_data_misses");
++	prefetch_metadata_misses = za_sysctl_read_stat("kstat.zfs.misc.arcstats.prefetch_metadata_misses");
++
++	hits   = za_sysctl_read_stat("kstat.zfs.misc.arcstats.hits");
++	misses = za_sysctl_read_stat("kstat.zfs.misc.arcstats.misses");
++
++	l2_size        = za_sysctl_read_stat("kstat.zfs.misc.arcstats.l2_size");
++	l2_read_bytes  = za_sysctl_read_stat("kstat.zfs.misc.arcstats.l2_read_bytes");
++	l2_write_bytes = za_sysctl_read_stat("kstat.zfs.misc.arcstats.l2_write_bytes");
++	l2_hits        = za_sysctl_read_stat("kstat.zfs.misc.arcstats.l2_hits");
++	l2_misses      = za_sysctl_read_stat("kstat.zfs.misc.arcstats.l2_misses");
++#endif
+ 
+ 
+ 	za_submit_size (arcsize, targetsize, minlimit, maxlimit);
+@@ -145,6 +188,7 @@
+ 
+ static int za_init (void) /* {{{ */
+ {
++#if HAVE_LIBKSTAT
+ 	ksp = NULL;
+ 
+ 	/* kstats chain already opened by update_kstat (using *kc), verify everything went fine. */
+@@ -153,6 +197,16 @@
+ 		ERROR ("zfs_arc plugin: kstat chain control structure not available.");
+ 		return (-1);
+ 	}
++#elif HAVE_SYSCTL
++	uint64_t arcsize = 0;
++	size_t arcsize_size = sizeof (arcsize);
++
++	if (sysctlbyname ("kstat.zfs.misc.arcstats.size", &arcsize, &arcsize_size, NULL, 0) < 0)
++	{
++		ERROR ("zfs_arc plugin: sysctl kstat.zfs.misc.arcstats structure not available.");
++		return (-1);
++	}
++#endif
+ 
+ 	return (0);
+ } /* }}} int za_init */
--- net-mgmt/collectd/pkg-plist.orig	2010-11-14 13:28:55.052837511 +1030
+++ net-mgmt/collectd/pkg-plist	2010-12-08 15:53:13.898232189 +1030
@@ -128,12 +128,15 @@
 include/collectd/client.h
 include/collectd/lcc_features.h
 libdata/pkgconfig/libcollectdclient.pc
 %%XMMS%%lib/collectd/xmms.a
 %%XMMS%%lib/collectd/xmms.la
 %%XMMS%%lib/collectd/xmms.so
+%%ZFS_ARC%%lib/collectd/zfs_arc.a
+%%ZFS_ARC%%lib/collectd/zfs_arc.la
+%%ZFS_ARC%%lib/collectd/zfs_arc.so
 %%DATADIR%%/postgresql_default.conf
 %%DATADIR%%/types.db
 @dirrm lib/collectd
 @dirrm include/collectd
 @dirrm %%DATADIR%%
 @exec mkdir -p /var/db/collectd


>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback 
State-Changed-By: edwin 
State-Changed-When: Thu Dec 9 00:40:16 UTC 2010 
State-Changed-Why:  
Awaiting maintainers feedback (via the GNATS Auto Assign Tool) 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152947 

From: Edwin Groothuis <edwin@FreeBSD.org>
To: ports@bsdserwis.com
Cc: bug-followup@FreeBSD.org
Subject: Re: ports/152947: Add ZFS ARC statistics into net-mgmt/collectd port
Date: Thu, 9 Dec 2010 00:40:14 UT

 Maintainer of net-mgmt/collectd,
 
 Please note that PR ports/152947 has just been submitted.
 
 If it contains a patch for an upgrade, an enhancement or a bug fix
 you agree on, reply to this email stating that you approve the patch
 and a committer will take care of it.
 
 The full text of the PR can be found at:
     http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/152947
 
 -- 
 Edwin Groothuis via the GNATS Auto Assign Tool
 edwin@FreeBSD.org

From: Krzysztof Stryjek <ports@bsdserwis.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: Re: ports/152947: Add ZFS ARC statistics into net-mgmt/collectd
 port
Date: Thu, 9 Dec 2010 09:34:48 +0100

 Hello,
 
 I approve this patch.
 
 Greetings,
 -- 
 Krzysztof Stryjek
 UNIX administrator/Juniper Networks Specialist
 email: wtp (at) bsdserwis (dot) com
 http://www.linkedin.com/in/KrzysztofStryjek
 GPG fingerprint: 8BD7 40CE 8994 0BBE CE6C  91CD 1292 8959 DC61 0E76
 
 In theory, there is no difference between theory and practice.
 In practice, there is.
State-Changed-From-To: feedback->open 
State-Changed-By: linimon 
State-Changed-When: Thu Dec 9 14:46:13 UTC 2010 
State-Changed-Why:  
Maintainer approved. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152947 
Responsible-Changed-From-To: freebsd-ports-bugs->miwi 
Responsible-Changed-By: miwi 
Responsible-Changed-When: Mon Jan 31 12:02:18 UTC 2011 
Responsible-Changed-Why:  
take 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152947 
State-Changed-From-To: open->feedback 
State-Changed-By: miwi 
State-Changed-When: Mon Jan 31 12:02:30 UTC 2011 
State-Changed-Why:  
-- net-mgmt/collectd/Makefile.orig      2010-12-07 10:02:42.958651026 
+1030 |+++ net-mgmt/collectd/Makefile 2010-12-09 10:40:18.041439748 
+1030 -------------------------- Patching file Makefile using Plan A... 
Hunk #1 failed at 13.  Hunk #2 succeeded at 37.  Hunk #3 succeeded at 
52.  Hunk #4 succeeded at 131.  Hunk #5 succeeded at 314.  1 out of 5 
hunks failed--saving rejects to Makefile.rej Hmm...  The next patch 
looks like a unified diff to me...  The text leading up to this was: 
-------------------------- 


patch reject can u please rework and resend to me as attachent. thx. 



http://www.freebsd.org/cgi/query-pr.cgi?pr=152947 
State-Changed-From-To: feedback->closed 
State-Changed-By: miwi 
State-Changed-When: Fri Mar 4 09:41:21 UTC 2011 
State-Changed-Why:  
submmiter timeout 

http://www.freebsd.org/cgi/query-pr.cgi?pr=152947 
>Unformatted:
