From wawa@shaper.yandex.ru  Tue Feb  5 07:04:15 2002
Return-Path: <wawa@shaper.yandex.ru>
Received: from shaper.yandex.ru (shaper.yandex.ru [213.180.193.102])
	by hub.freebsd.org (Postfix) with ESMTP id F0D1437B41F
	for <FreeBSD-gnats-submit@freebsd.org>; Tue,  5 Feb 2002 07:04:13 -0800 (PST)
Received: (from wawa@localhost)
	by shaper.yandex.ru (8.11.3/8.11.3) id g15ErJM09362;
	Tue, 5 Feb 2002 17:53:19 +0300 (MSK)
	(envelope-from wawa)
Message-Id: <200202051453.g15ErJM09362@shaper.yandex.ru>
Date: Tue, 5 Feb 2002 17:53:19 +0300 (MSK)
From: wawa@yandex-team.ru
Reply-To: wawa@yandex-team.ru
To: FreeBSD-gnats-submit@freebsd.org
Cc:
Subject: IPFW skipto works too slow
X-Send-Pr-Version: 3.113
X-GNATS-Notify:

>Number:         34639
>Category:       kern
>Synopsis:       IPFW skipto works too slow
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Feb 05 07:10:01 PST 2002
>Closed-Date:    Wed Feb 20 09:16:32 PST 2002
>Last-Modified:  Wed Feb 20 09:18:00 PST 2002
>Originator:     Vladimir Ivanov
>Release:        FreeBSD 5.0-CURRENT i386
>Organization:
OOO Yandex
>Environment:
System: FreeBSD shaper.yandex.ru 5.0-CURRENT FreeBSD 5.0-CURRENT #3: Mon Feb 4 13:34:01 MSK 2002 wawa@shaper.yandex.ru:/usr/src/sys/compile/TEST i386


>Description:
The original implementation of skipto rule use brute-force to find the appropriate rule. 
The suggested implementation use indexed access.
>How-To-Repeat:
>Fix:

--- ip_fw.c.original	Fri Mar  9 11:13:08 2001
+++ ip_fw.c.hacked_by_wawa	Tue Feb  5 17:44:56 2002
@@ -108,6 +108,8 @@
 SYSCTL_INT(_net_inet_ip_fw, OID_AUTO, verbose_limit, CTLFLAG_RW, 
     &fw_verbose_limit, 0, "Set upper limit of matches of ipfw rules logged");
 
+static struct ip_fw_chain *skiptohash[IPFW_DEFAULT_RULE];
+
 /*
  * Extension for stateful ipfw.
  *
@@ -903,9 +905,8 @@
     int rule = me->rule->fw_skipto_rule ; /* guess... */
 
     if ( (me->rule->fw_flg & IP_FW_F_COMMAND) == IP_FW_F_SKIPTO )
-	for (chain = LIST_NEXT(me,next); chain ; chain = LIST_NEXT(chain,next))
-	    if (chain->rule->fw_number >= rule)
-                return chain ;
+	return skiptohash[rule];
+
     return LIST_NEXT(me,next) ; /* failure or not a skipto */
 }
 
@@ -1056,8 +1057,8 @@
 		if (skipto != 0) {
 			if (skipto >= IPFW_DEFAULT_RULE)
 				goto dropit;
-			while (chain && chain->rule->fw_number <= skipto)
-				chain = LIST_NEXT(chain, next);
+			chain = skiptohash[skipto];
+
 			if (chain == NULL)
 				goto dropit;
 		}
@@ -1528,6 +1529,8 @@
 			nbr += 100;
 		ftmp->fw_number = frwl->fw_number = nbr;
 	}
+	/* save the pointer in hash */
+	skiptohash[ftmp->fw_number] = fwc;
 
 	/* Got a valid number; now insert it, keeping the list ordered */
 	LIST_FOREACH(fcp, chainptr, next) {
@@ -1561,6 +1564,7 @@
 
 				/* prevent access to rules while removing them */
 				s = splnet();
+				skiptohash[number] = NULL;
 				while (fcp && fcp->rule->fw_number == number) {
 					struct ip_fw_chain *next;
 
@@ -1897,6 +1901,7 @@
 	case IP_FW_FLUSH:
 		s = splnet();
 		remove_dyn_rule(NULL, 1 /* force delete */);
+		memset(skiptohash,0,sizeof(skiptohash));
 		splx(s);
 		while ( (fcp = LIST_FIRST(&ip_fw_chain_head)) &&
 		     fcp->rule->fw_number != IPFW_DEFAULT_RULE ) {
@@ -2044,6 +2049,7 @@
 		return 0;
 	case MOD_UNLOAD:
 		s = splnet();
+		memset(skiptohash, 0, sizeof(skiptohash));
 		ip_fw_chk_ptr =  old_chk_ptr;
 		ip_fw_ctl_ptr =  old_ctl_ptr;
 		remove_dyn_rule(NULL, 1 /* force delete */);
>Release-Note:
>Audit-Trail:

From: Peter Pentchev <roam@ringlet.net>
To: wawa@yandex-team.ru
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/34639: IPFW skipto works too slow
Date: Tue, 5 Feb 2002 17:22:42 +0200

 On Tue, Feb 05, 2002 at 05:53:19PM +0300, wawa@yandex-team.ru wrote:
 > 
 > >Number:         34639
 > >Category:       kern
 > >Synopsis:       IPFW skipto works too slow
 > >Description:
 > The original implementation of skipto rule use brute-force to find the appropriate rule. 
 > The suggested implementation use indexed access.
 
 Your suggested implementation has just one drawback: it assumes that
 the rule exists.  AFAIR, the current skipto implementation will skip
 to the lowest-numbered rule no lower than the specified one, so you
 can safely delete the rule with that specific number and still rest
 assured that the following rules will be honored.
 
 G'luck,
 Peter
 
 -- 
 Peter Pentchev	roam@ringlet.net	roam@FreeBSD.org
 PGP key:	http://people.FreeBSD.org/~roam/roam.key.asc
 Key fingerprint	FDBA FD79 C26F 3C51 C95E  DF9E ED18 B68D 1619 4553
 This sentence was in the past tense.

From: Vladimir Ivanov <wawa@yandex-team.ru>
To: Peter Pentchev <roam@ringlet.net>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/34639: IPFW skipto works too slow
Date: Tue, 05 Feb 2002 20:03:17 +0300

 Peter Pentchev wrote:
 
 > On Tue, Feb 05, 2002 at 05:53:19PM +0300, wawa@yandex-team.ru wrote:
 > 
 >>>Number:         34639
 >>>Category:       kern
 >>>Synopsis:       IPFW skipto works too slow
 >>>Description:
 >>>
 >>The original implementation of skipto rule use brute-force to find the appropriate rule. 
 >>The suggested implementation use indexed access.
 >>
 > 
 > Your suggested implementation has just one drawback: it assumes that
 > the rule exists.  AFAIR, the current skipto implementation will skip
 > to the lowest-numbered rule no lower than the specified one, so you
 > can safely delete the rule with that specific number and still rest
 > assured that the following rules will be honored.
 
 
 Yes you're right. But my idea was to make it more intuitive. The 
 domcumented behaviour can be achived by a couple lines of code which 
 can be placed in the ad_ and del_ procedure but not in chk_ . We can 
 just fill the index with duplicated values. Doesn't it ? It's important 
 because it saves running time significantly still.
 
 
 > 
 > G'luck,
 > Peter
 > 
 > 
 
 Truly,
 
 -- 
 Vladimir Ivanov
 OOO "Yandex"
 t. +7 095 974-3555
 f. +7 095 974-3565
 @: noc@yandex.net (corporate)
     wawa@yandex-team.ru (personal)
 www: yandex.ru
 --
 My opinions may have changed, but not the fact that I am right.
 

From: Peter Pentchev <roam@ringlet.net>
To: Vladimir Ivanov <wawa@yandex-team.ru>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/34639: IPFW skipto works too slow
Date: Tue, 5 Feb 2002 19:10:45 +0200

 On Tue, Feb 05, 2002 at 08:03:17PM +0300, Vladimir Ivanov wrote:
 > Peter Pentchev wrote:
 > 
 > > On Tue, Feb 05, 2002 at 05:53:19PM +0300, wawa@yandex-team.ru wrote:
 > > 
 > >>>Number:         34639
 > >>>Category:       kern
 > >>>Synopsis:       IPFW skipto works too slow
 > >>>Description:
 > >>>
 > >>The original implementation of skipto rule use brute-force to find the appropriate rule. 
 > >>The suggested implementation use indexed access.
 > >>
 > > 
 > > Your suggested implementation has just one drawback: it assumes that
 > > the rule exists.  AFAIR, the current skipto implementation will skip
 > > to the lowest-numbered rule no lower than the specified one, so you
 > > can safely delete the rule with that specific number and still rest
 > > assured that the following rules will be honored.
 > 
 > 
 > Yes you're right. But my idea was to make it more intuitive. The 
 > domcumented behaviour can be achived by a couple lines of code which 
 > can be placed in the ad_ and del_ procedure but not in chk_ . We can 
 > just fill the index with duplicated values. Doesn't it ? It's important 
 > because it saves running time significantly still.
 
 I said 'just one drawback'; this implies that I like your change :)
 Right now I do not have the opportunity to look at the ipfw code and
 come up with patches, but yes, a couple of lines in adding and deleting
 rules should be enough.
 
 Keep up the good work, and hopefully somebody who knows their way
 around ipfw will notice this and commit it after suitable refinement :)
 
 G'luck,
 Peter
 
 -- 
 Peter Pentchev	roam@ringlet.net	roam@FreeBSD.org
 PGP key:	http://people.FreeBSD.org/~roam/roam.key.asc
 Key fingerprint	FDBA FD79 C26F 3C51 C95E  DF9E ED18 B68D 1619 4553
 This sentence every third, but it still comprehensible.

From: Bill Fumerola <billf@mu.org>
To: wawa@yandex-team.ru
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/34639: IPFW skipto works too slow
Date: Tue, 5 Feb 2002 21:31:03 -0800

 On Tue, Feb 05, 2002 at 05:53:19PM +0300, wawa@yandex-team.ru wrote:
 
 > >Description:
 > The original implementation of skipto rule use brute-force to find the appropriate rule. 
 > The suggested implementation use indexed access.
 
 the ipfw code caches the skipto rule the first time it gets used. i don't
 think cacheing it in a different spot really helps anything.
 
 are you actually seeing performance problems? can you actually prove
 that ipfw is faster with this change?
 
 -- 
 - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org / billf@mu.org
 - my anger management counselor can beat up your self-affirmation therapist
 
 

From: Vladimir Ivanov <wawa@yandex-team.ru>
To: Bill Fumerola <billf@mu.org>
Cc: FreeBSD-gnats-submit@freebsd.org
Subject: Re: kern/34639: IPFW skipto works too slow
Date: Wed, 06 Feb 2002 09:07:34 +0300

 Bill Fumerola wrote:
 
 > On Tue, Feb 05, 2002 at 05:53:19PM +0300, wawa@yandex-team.ru wrote:
 > 
 > 
 >>>Description:
 >>>
 >>The original implementation of skipto rule use brute-force to find the appropriate rule. 
 >>The suggested implementation use indexed access.
 >>
 > 
 > the ipfw code caches the skipto rule the first time it gets used. i don't
 > think cacheing it in a different spot really helps anything.
 
 
 I've never seen caching of skipto's in the code. Maybe the cacheing is 
 very tricky. ;-)
 
 
 > 
 > are you actually seeing performance problems? can you actually prove
 > that ipfw is faster with this change?
 > 
 > 
 
 Sure, I've tested it on a real system. The system was unable to process more than several hundred rules with 
 
 skipto's. It was losing packets. 'top' utility reported about 99% CPU for 'interrupts' and the 
 
 total bandwidth degradated twice after adding even only skipto rule in the tail of chain. The behaviour after 
 
 applying patch is expected. 
 
 
 Best regards,
 
 -- 
 Vladimir Ivanov
 Network Operation Center
 OOO "Yandex"
 e-mail: noc@yandex.net (corporate)
          wawa@yandex-team.ru (personal)
 www: www.yandex.ru
 tel: +7 095 974-3555
 fax: +7 095 974-3565
 --
 A diplomat is someone who can tell you to go to hell in such a way that
 you will look forward to the trip.
 

From: Luigi Rizzo <rizzo@icir.org>
To: freebsd-gnats-submit@freebsd.org
Cc: wawa@yandex.team.ru, billf@mu.org
Subject: Re: kern/34639: IPFW skipto works too slow
Date: Wed, 20 Feb 2002 08:25:22 -0800

 Vladimir is indeed right, the code to cache skipto targets
 somehow is not there!
 A fix is attached below, I should commit it briefly as
 I have a chance to test it.
 
 thanks for the report
 
 	luigi
 
 
 > lcvs diff -u ip_fw.c
 Index: ip_fw.c
 ===================================================================
 RCS file: /home/xorpc/u2/freebsd/src/sys/netinet/ip_fw.c,v
 retrieving revision 1.180
 diff -u -r1.180 ip_fw.c
 --- ip_fw.c     10 Feb 2002 22:22:05 -0000      1.180
 +++ ip_fw.c     20 Feb 2002 16:05:38 -0000
 @@ -1512,11 +1512,13 @@
                         return(f->fw_divert_port | IP_FW_PORT_TEE_FLAG);
  #endif
                 case IP_FW_F_SKIPTO: /* XXX check */
 -                       f = f->next_rule_ptr ? f->next_rule_ptr :
 -                               lookup_next_rule(f) ;
 +                       if (f->next_rule_ptr == NULL)
 +                           f->next_rule_ptr = lookup_next_rule(f) ;
 +                       f = f->next_rule_ptr;
                         if (!f)
                             goto dropit;
                         goto again ;
 +
                 case IP_FW_F_PIPE:
                 case IP_FW_F_QUEUE:
                         *flow_id = f; /* XXX set flow id */
 
State-Changed-From-To: open->closed 
State-Changed-By: luigi 
State-Changed-When: Wed Feb 20 09:16:32 PST 2002 
State-Changed-Why:  
Committed the last patch, which fixes the problem. 


http://www.FreeBSD.org/cgi/query-pr.cgi?pr=34639 
>Unformatted:
