From nobody@FreeBSD.org  Thu Apr 10 13:25:03 2014
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by hub.freebsd.org (Postfix) with ESMTPS id 4995DFD7
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 10 Apr 2014 13:25:03 +0000 (UTC)
Received: from cgiserv.freebsd.org (cgiserv.freebsd.org [IPv6:2001:1900:2254:206a::50:4])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 1C00112AF
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 10 Apr 2014 13:25:03 +0000 (UTC)
Received: from cgiserv.freebsd.org ([127.0.1.6])
	by cgiserv.freebsd.org (8.14.8/8.14.8) with ESMTP id s3ADP2Yk055141
	for <freebsd-gnats-submit@FreeBSD.org>; Thu, 10 Apr 2014 13:25:02 GMT
	(envelope-from nobody@cgiserv.freebsd.org)
Received: (from nobody@localhost)
	by cgiserv.freebsd.org (8.14.8/8.14.8/Submit) id s3ADP2Ll055136;
	Thu, 10 Apr 2014 13:25:02 GMT
	(envelope-from nobody)
Message-Id: <201404101325.s3ADP2Ll055136@cgiserv.freebsd.org>
Date: Thu, 10 Apr 2014 13:25:02 GMT
From: Karim Fodil-Lemelin <fodillemlinkarim@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: ng_callout() timeouts trigger packets queuing and out of order packets
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         188421
>Category:       kern
>Synopsis:       [libnetgraph] [patch] ng_callout() timeouts trigger packets queuing and out of order packets
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-net
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Apr 10 13:30:00 UTC 2014
>Closed-Date:    
>Last-Modified:  Wed Apr 16 01:37:54 UTC 2014
>Originator:     Karim Fodil-Lemelin
>Release:        FreeBSD 7, FreeBSD 9
>Organization:
XipLink Inc.
>Environment:
FreeBSD pluto 7.4-RELEASE FreeBSD 7.4-RELEASE #0: Mon Apr 15 20:48:02 UTC 2013     kfl@pluto:/usr/src/sys/amd64/compile/SIMULATOR  amd64

>Description:
I have seen an issue in netgraph
where, if called, a callout routine registered by ng_callout() will trigger
packet queuing inside the worklist of netgraph since ng_callout() makes my
node suddenly a WRITER node (therefore non reentrant) for the duration of
the call.

So as soon as the callout function returns, all following packets will get
directly passed to the node again and when the ngintr thread gets executed
then only then will I get the queued packets. This introduces out of order
packets in the flow. I am using the current patch below to solve the issue
and I am wondering if there is anything wrong with it (and maybe contribute
back :).

>How-To-Repeat:
Have a constant stream of packets sent to a netgraph node. The node must at
some point have registered a timeout function. Lets have it exceute every
10 miliseconds. Track the order (looking at ip id) of the packets as they
come out. You should see out of order packets whenever the callout function
is called.
>Fix:
Use the provided patch.

Patch attached with submission follows:

@@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p
hook, int ticks,

        if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL)
                return (ENOMEM);

-       item->el_flags |= NGQF_WRITER;
+       if ((node->nd_flags & NGF_FORCE_WRITER) ||
+           (hook && (hook->hk_flags & HK_FORCE_WRITER)))

+         item->el_flags |= NGQF_WRITER;
+       else

+         item->el_flags |= NGQF_READER;
+
        NG_NODE_REF(node);              /* and one for the item */
        NGI_SET_NODE(item, node);
        if (hook) {


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Wed Apr 16 01:36:26 UTC 2014 
Responsible-Changed-Why:  
reclassify. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=188421 
>Unformatted:
