From nobody@FreeBSD.org  Fri Apr 15 14:10:51 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id B3B0C106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 15 Apr 2011 14:10:51 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id A226F8FC19
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 15 Apr 2011 14:10:51 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p3FEAoxN052497
	for <freebsd-gnats-submit@FreeBSD.org>; Fri, 15 Apr 2011 14:10:50 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p3FEAo3l052496;
	Fri, 15 Apr 2011 14:10:50 GMT
	(envelope-from nobody)
Message-Id: <201104151410.p3FEAo3l052496@red.freebsd.org>
Date: Fri, 15 Apr 2011 14:10:50 GMT
From: Balazs Scheidler <bazsi@balabit.hu>
To: freebsd-gnats-submit@FreeBSD.org
Subject: Please add kqueue support for /dev/klog
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         156423
>Category:       kern
>Synopsis:       [kqueue] [patch] Please add kqueue support for /dev/klog
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    kib
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Fri Apr 15 14:20:03 UTC 2011
>Closed-Date:    Wed Feb 08 15:22:09 UTC 2012
>Last-Modified:  Wed Feb 08 15:22:09 UTC 2012
>Originator:     Balazs Scheidler
>Release:        8.1-RELEASE
>Organization:
BalaBit IT Ltd.
>Environment:
FreeBSD fbsd81.balabit 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:

The multi-threaded version of syslog-ng (an alternative to syslogd) would try to use the kqueue mechanism to drive its main loop for maximum performance. 

However /dev/klog currently doesn't support kqueue().

There's a patch by DragonflyBSD, which seems to implement it (however I don't know FreeBSD kernel code that much):

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/fef8f01a63adb168dd043d0e835694b0d02c59c9

The kqueue() support is implemented through a wrapper library called libivykis (http://sourceforge.net/projects/libivykis/). This library makes it possible to enable or disable polling mechanisms at runtime using an environment variable.

For now, I'll simply disable kqueue() support, and revert to using plain poll(), since with the current syslog-ng architecture it's either an all-or-nothing decision. (e.g. either everything goes with kqueue() or nothing) 

I'm sure sockets would work nicely and users may come up with alternatives (like running two instances of syslog-ng, one for /dev/klog the other for the rest), but it's a big hassle.

So if it's not a big issue, please try to support it in one of the upcoming FreeBSD versions, so I can reenable kqueue() support.

Thanks.

>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:

From: Mateusz Guzik <mjguzik@gmail.com>
To: bug-followup@FreeBSD.org, bazsi@balabit.hu
Cc:  
Subject: Re: kern/156423: [kqueue] [patch] Please add kqueue support for
 /dev/klog
Date: Tue, 31 Jan 2012 00:02:05 +0000

 Can you please try this:
 
 http://student.agh.edu.pl/~mjguzik/patches/dev_klog-kqueue.patch
 
 Tested with this trivial program:
 http://student.agh.edu.pl/~mjguzik/kqread.c
 
 This code is mostly stolen from audit pipe implementation, patch from
 dragonfly people was not really helpful. :)
 
 -- 
 Mateusz Guzik <mjguzik gmail.com>

From: Balazs Scheidler <bazsi@balabit.hu>
To: Mateusz Guzik <mjguzik@gmail.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/156423: [kqueue] [patch] Please add kqueue support for
 /dev/klog
Date: Tue, 31 Jan 2012 15:03:05 +0100

 On Tue, 2012-01-31 at 00:02 +0000, Mateusz Guzik wrote:
 > Can you please try this:
 > 
 > http://student.agh.edu.pl/~mjguzik/patches/dev_klog-kqueue.patch
 > 
 > Tested with this trivial program:
 > http://student.agh.edu.pl/~mjguzik/kqread.c
 > 
 > This code is mostly stolen from audit pipe implementation, patch from
 > dragonfly people was not really helpful. :)
 > 
 
 I'm afraid I don't know FreeBSD enough to give this a spin. Compiling a
 kernel is way above my expertise on that platform.
 
 Can I forward this email to the syslog-ng mailing list? I definitely
 know there are users of syslog-ng on FreeBSD, who should know better
 than me.
 
 -- 
 Bazsi
 
 
Responsible-Changed-From-To: freebsd-bugs->grab 
Responsible-Changed-By: kib 
Responsible-Changed-When: Wed Feb 1 14:15:04 UTC 2012 
Responsible-Changed-Why:  


http://www.freebsd.org/cgi/query-pr.cgi?pr=156423 
Responsible-Changed-From-To: grab->kib 
Responsible-Changed-By: kib 
Responsible-Changed-When: Wed Feb 1 14:15:22 UTC 2012 
Responsible-Changed-Why:  
Grab. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=156423 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/156423: commit references a PR
Date: Wed,  1 Feb 2012 14:35:11 +0000 (UTC)

 Author: kib
 Date: Wed Feb  1 14:34:52 2012
 New Revision: 230866
 URL: http://svn.freebsd.org/changeset/base/230866
 
 Log:
   Add kqueue support to /dev/klog.
   
   Submitted by:	Mateusz Guzik <mjguzik gmail com>
   PR:	  kern/156423
   MFC after:	1 weeks
 
 Modified:
   head/sys/kern/subr_log.c
 
 Modified: head/sys/kern/subr_log.c
 ==============================================================================
 --- head/sys/kern/subr_log.c	Wed Feb  1 13:39:52 2012	(r230865)
 +++ head/sys/kern/subr_log.c	Wed Feb  1 14:34:52 2012	(r230866)
 @@ -59,6 +59,7 @@ static	d_close_t	logclose;
  static	d_read_t	logread;
  static	d_ioctl_t	logioctl;
  static	d_poll_t	logpoll;
 +static	d_kqfilter_t	logkqfilter;
  
  static	void logtimeout(void *arg);
  
 @@ -69,9 +70,20 @@ static struct cdevsw log_cdevsw = {
  	.d_read =	logread,
  	.d_ioctl =	logioctl,
  	.d_poll =	logpoll,
 +	.d_kqfilter =	logkqfilter,
  	.d_name =	"log",
  };
  
 +static int	logkqread(struct knote *note, long hint);
 +static void	logkqdetach(struct knote *note);
 +
 +static struct filterops log_read_filterops = {
 +	.f_isfd =	1,
 +	.f_attach =	NULL,
 +	.f_detach =	logkqdetach,
 +	.f_event =	logkqread,
 +};
 +
  static struct logsoftc {
  	int	sc_state;		/* see above for possibilities */
  	struct	selinfo sc_selp;	/* process waiting on select call */
 @@ -181,6 +193,40 @@ logpoll(struct cdev *dev, int events, st
  	return (revents);
  }
  
 +static int
 +logkqfilter(struct cdev *dev, struct knote *kn)
 +{
 +
 +	if (kn->kn_filter != EVFILT_READ)
 +		return (EINVAL);
 +
 +	kn->kn_fop = &log_read_filterops;
 +	kn->kn_hook = NULL;
 +
 +	mtx_lock(&msgbuf_lock);
 +	knlist_add(&logsoftc.sc_selp.si_note, kn, 1);
 +	mtx_unlock(&msgbuf_lock);
 +	return (0);
 +}
 +
 +static int
 +logkqread(struct knote *kn, long hint)
 +{
 +
 +	mtx_assert(&msgbuf_lock, MA_OWNED);
 +	kn->kn_data = msgbuf_getcount(msgbufp);
 +	return (kn->kn_data != 0);
 +}
 +
 +static void
 +logkqdetach(struct knote *kn)
 +{
 +
 +	mtx_lock(&msgbuf_lock);
 +	knlist_remove(&logsoftc.sc_selp.si_note, kn, 1);
 +	mtx_unlock(&msgbuf_lock);
 +}
 +
  static void
  logtimeout(void *arg)
  {
 @@ -198,6 +244,7 @@ logtimeout(void *arg)
  	}
  	msgbuftrigger = 0;
  	selwakeuppri(&logsoftc.sc_selp, LOG_RDPRI);
 +	KNOTE_LOCKED(&logsoftc.sc_selp.si_note, 0);
  	if ((logsoftc.sc_state & LOG_ASYNC) && logsoftc.sc_sigio != NULL)
  		pgsigio(&logsoftc.sc_sigio, SIGIO, 0);
  	cv_broadcastpri(&log_wakeup, LOG_RDPRI);
 @@ -256,6 +303,7 @@ log_drvinit(void *unused)
  
  	cv_init(&log_wakeup, "klog");
  	callout_init_mtx(&logsoftc.sc_callout, &msgbuf_lock, 0);
 +	knlist_init_mtx(&logsoftc.sc_selp.si_note, &msgbuf_lock);
  	make_dev_credf(MAKEDEV_ETERNAL, &log_cdevsw, 0, NULL, UID_ROOT,
  	    GID_WHEEL, 0600, "klog");
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 

From: dfilter@FreeBSD.ORG (dfilter service)
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/156423: commit references a PR
Date: Wed,  8 Feb 2012 14:21:31 +0000 (UTC)

 Author: kib
 Date: Wed Feb  8 14:21:20 2012
 New Revision: 231203
 URL: http://svn.freebsd.org/changeset/base/231203
 
 Log:
   MFC r230866:
   Add kqueue support to /dev/klog.
   
   PR:	kern/156423
 
 Modified:
   stable/9/sys/kern/subr_log.c
 Directory Properties:
   stable/9/sys/   (props changed)
 
 Modified: stable/9/sys/kern/subr_log.c
 ==============================================================================
 --- stable/9/sys/kern/subr_log.c	Wed Feb  8 14:09:08 2012	(r231202)
 +++ stable/9/sys/kern/subr_log.c	Wed Feb  8 14:21:20 2012	(r231203)
 @@ -59,6 +59,7 @@ static	d_close_t	logclose;
  static	d_read_t	logread;
  static	d_ioctl_t	logioctl;
  static	d_poll_t	logpoll;
 +static	d_kqfilter_t	logkqfilter;
  
  static	void logtimeout(void *arg);
  
 @@ -69,9 +70,20 @@ static struct cdevsw log_cdevsw = {
  	.d_read =	logread,
  	.d_ioctl =	logioctl,
  	.d_poll =	logpoll,
 +	.d_kqfilter =	logkqfilter,
  	.d_name =	"log",
  };
  
 +static int	logkqread(struct knote *note, long hint);
 +static void	logkqdetach(struct knote *note);
 +
 +static struct filterops log_read_filterops = {
 +	.f_isfd =	1,
 +	.f_attach =	NULL,
 +	.f_detach =	logkqdetach,
 +	.f_event =	logkqread,
 +};
 +
  static struct logsoftc {
  	int	sc_state;		/* see above for possibilities */
  	struct	selinfo sc_selp;	/* process waiting on select call */
 @@ -181,6 +193,40 @@ logpoll(struct cdev *dev, int events, st
  	return (revents);
  }
  
 +static int
 +logkqfilter(struct cdev *dev, struct knote *kn)
 +{
 +
 +	if (kn->kn_filter != EVFILT_READ)
 +		return (EINVAL);
 +
 +	kn->kn_fop = &log_read_filterops;
 +	kn->kn_hook = NULL;
 +
 +	mtx_lock(&msgbuf_lock);
 +	knlist_add(&logsoftc.sc_selp.si_note, kn, 1);
 +	mtx_unlock(&msgbuf_lock);
 +	return (0);
 +}
 +
 +static int
 +logkqread(struct knote *kn, long hint)
 +{
 +
 +	mtx_assert(&msgbuf_lock, MA_OWNED);
 +	kn->kn_data = msgbuf_getcount(msgbufp);
 +	return (kn->kn_data != 0);
 +}
 +
 +static void
 +logkqdetach(struct knote *kn)
 +{
 +
 +	mtx_lock(&msgbuf_lock);
 +	knlist_remove(&logsoftc.sc_selp.si_note, kn, 1);
 +	mtx_unlock(&msgbuf_lock);
 +}
 +
  static void
  logtimeout(void *arg)
  {
 @@ -198,6 +244,7 @@ logtimeout(void *arg)
  	}
  	msgbuftrigger = 0;
  	selwakeuppri(&logsoftc.sc_selp, LOG_RDPRI);
 +	KNOTE_LOCKED(&logsoftc.sc_selp.si_note, 0);
  	if ((logsoftc.sc_state & LOG_ASYNC) && logsoftc.sc_sigio != NULL)
  		pgsigio(&logsoftc.sc_sigio, SIGIO, 0);
  	cv_broadcastpri(&log_wakeup, LOG_RDPRI);
 @@ -256,6 +303,7 @@ log_drvinit(void *unused)
  
  	cv_init(&log_wakeup, "klog");
  	callout_init_mtx(&logsoftc.sc_callout, &msgbuf_lock, 0);
 +	knlist_init_mtx(&logsoftc.sc_selp.si_note, &msgbuf_lock);
  	make_dev_credf(MAKEDEV_ETERNAL, &log_cdevsw, 0, NULL, UID_ROOT,
  	    GID_WHEEL, 0600, "klog");
  }
 _______________________________________________
 svn-src-all@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/svn-src-all
 To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"
 
State-Changed-From-To: open->closed 
State-Changed-By: kib 
State-Changed-When: Wed Feb 8 15:21:48 UTC 2012 
State-Changed-Why:  
Merged to 9. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=156423 
>Unformatted:
