From nobody@FreeBSD.org  Tue Mar  7 18:19:09 2006
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id D639416A420
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  7 Mar 2006 18:19:09 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [216.136.204.117])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 765B343D64
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  7 Mar 2006 18:19:09 +0000 (GMT)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k27IJ9ff019266
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 7 Mar 2006 18:19:09 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.13.1/8.13.1/Submit) id k27IJ9B9019263;
	Tue, 7 Mar 2006 18:19:09 GMT
	(envelope-from nobody)
Message-Id: <200603071819.k27IJ9B9019263@www.freebsd.org>
Date: Tue, 7 Mar 2006 18:19:09 GMT
From: Emil Cazamir <enterco@yahoo.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: altq support for vlan driver
X-Send-Pr-Version: www-2.3

>Number:         94182
>Category:       kern
>Synopsis:       [altq] [request] altq support for vlan driver
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          suspended
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Tue Mar 07 18:20:05 GMT 2006
>Closed-Date:    
>Last-Modified:  Mon Jan 28 09:14:43 UTC 2008
>Originator:     Emil Cazamir
>Release:        5.4-STABLE
>Organization:
none
>Environment:
FreeBSD r1 5.4-STABLE FreeBSD 5.4-STABLE #6: Mon Jan 16 13:12:19 EET 2006     root@r1:/usr/obj/usr/src/sys/BC  i386

>Description:
From altq(4):

"SUPPORTED DEVICES
     The driver modifications described in altq(9) are required to use a cer-
     tain network card with ALTQ.  They have been applied to the following
     hardware drivers: an(4), ath(4), awi(4), bfe(4), dc(4), de(4), ed(4),
     em(4), fxp(4), hme(4), lnc(4), re(4), rl(4), sf(4), sis(4), sk(4), vr(4),
     wi(4), and xl(4)."

At this time there is no support for altq using vlan interface. I use the following conficuration:
 physical interface: bge
    vlan1: vlan tag 1 vlandev bge0
    vlan2: vlan tag 2 vlandev bge0

I would like to use a setup like following:
- pf.conf -
  altq on vlan1 ...params...
  altq on vlan2 ...params...

>How-To-Repeat:
enable pf and altq and try to use "altq on vlan" statements into /etc/pf.conf.
>Fix:
perhaps some modifications to altq driver would do the thing, but i can't do them.
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->suspended 
State-Changed-By: linimon 
State-Changed-When: Tue Mar 7 21:15:44 UTC 2006 
State-Changed-Why:  
Mark suspended awaiting someone to take an interest in this. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=94182 

From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Emil Cazamir <enterco@yahoo.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/94182: altq support for vlan driver
Date: Wed, 15 Mar 2006 18:47:53 +0300

   Emil,
 
   if you search through mail archives, you will find, that the
 question about ALTQ on vlan(4) was raised several times. There is
 an opinion, that the idea itself is not correct at all.
 
 -- 
 Totus tuus, Glebius.
 GLEBIUS-RIPN GLEB-RIPE

From: Emil Cazamir <enterco@yahoo.com>
To: Gleb Smirnoff <glebius@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/94182: altq support for vlan driver
Date: Tue, 21 Mar 2006 11:50:26 -0800 (PST)

 --0-1856313939-1142970626=:89149
 Content-Type: text/plain; charset=iso-8859-1
 Content-Transfer-Encoding: 8bit
 
 Gleb Smirnoff <glebius@FreeBSD.org> wrote:   Emil,
 
   if you search through mail archives, you will find, that the
 question about ALTQ on vlan(4) was raised several times. There is
 an opinion, that the idea itself is not correct at all.
 
 -- 
 Totus tuus, Glebius.
 GLEBIUS-RIPN GLEB-RIPE
 
 I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: 
 - physical device:
    |- vlan1 - cbq on vlan1
    |- vlan2 - priq on vlan2
    `- vlan3 - hfsc on vlan3
 At this time i know that altq can make use of only one traffic discipline on an interface, which makes the above case only a wish.
 I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.
 
 Best regards,
 Emil Cazamir
 
 
 		
 ---------------------------------
 Yahoo! Travel
  Find  great deals to the top 10 hottest destinations!
 --0-1856313939-1142970626=:89149
 Content-Type: text/html; charset=iso-8859-1
 Content-Transfer-Encoding: 8bit
 
 <b><i>Gleb Smirnoff &lt;glebius@FreeBSD.org&gt;</i></b> wrote:<blockquote class="replbq" style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;">   Emil,<br><br>  if you search through mail archives, you will find, that the<br>question about ALTQ on vlan(4) was raised several times. There is<br>an opinion, that the idea itself is not correct at all.<br><br>-- <br>Totus tuus, Glebius.<br>GLEBIUS-RIPN GLEB-RIPE<br></blockquote><br>I'm not the person to decide if ALTQ and vlan(4)  is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: <br>- physical device:<br>&nbsp;&nbsp; |- vlan1 - cbq on vlan1<br>&nbsp;&nbsp; |- vlan2 - priq on vlan2<br>&nbsp;&nbsp; `- vlan3 - hfsc on vlan3<br>At this time i know that altq can make use of only one traffic discipline on an
  interface, which makes the above case only a wish.<br>I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.<br><br>Best  regards,<br>Emil Cazamir<br><br><p>
 		<hr size=1>Yahoo! Travel<br> 
 <a href="http://us.lrd.yahoo.com/_ylc=X3oDMTFscDlocTFiBF9TAzMyOTc1MDIEX3MDMjcxOTQ4MQRwb3MDMgRzZWMDbWFpbC1mb290ZXIEc2xrA3l0LXR0/SIG=12hqieud9/**http%3a//leisure.travelocity.com/Promotions/0,,YHOE%7c1381%7cvacs_main,00.html">Find  
 great deals</a> to the top 10 hottest destinations!
 --0-1856313939-1142970626=:89149--

From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Emil Cazamir <enterco@yahoo.com>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/94182: altq support for vlan driver
Date: Wed, 22 Mar 2006 14:38:12 +0300

   Emil,
 
 On Tue, Mar 21, 2006 at 11:50:26AM -0800, Emil Cazamir wrote:
 E> I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: 
 E> - physical device:
 E>    |- vlan1 - cbq on vlan1
 E>    |- vlan2 - priq on vlan2
 E>    `- vlan3 - hfsc on vlan3
 
 This may not work, unless you put a bandwidth limit on each vlan interface,
 that is equal to limit of parent interface / number of vlans.
 
 E> At this time i know that altq can make use of only one traffic discipline on an interface, which makes the above case only a wish.
 E> I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.
 
 The first (theoretical) obstacle is that ALTQ is designed to shape traffic on
 an interface, where exists a *bandwidth limit* and a *queue*. These two
 things are property of a physical inteface. The vlan(4) interface doesn't
 have any bandwidth limit. The current implementation has a queue. Packets
 are temporarily queued, before they are sent to the underlying physical
 driver. We have a plan to remove this queue, since it is a performance loss
 for vlan(4) driver.
 
 The second (practical) problem is that after removal of the intermediate
 queue in the vlan(4) driver it will be difficult to add ALTQ support. No
 queueing - no ALTernate Queueing.
 
 -- 
 Totus tuus, Glebius.
 GLEBIUS-RIPN GLEB-RIPE

From: Emil Cazamir <enterco@yahoo.com>
To: Gleb Smirnoff <glebius@FreeBSD.org>
Cc: freebsd-gnats-submit@FreeBSD.org
Subject: Re: kern/94182: altq support for vlan driver
Date: Thu, 23 Mar 2006 10:20:44 -0800 (PST)

 --0-980750068-1143138044=:59095
 Content-Type: text/plain; charset=iso-8859-1
 Content-Transfer-Encoding: 8bit
 
 Gleb Smirnoff <glebius@FreeBSD.org> wrote:   Emil,
 
 On Tue, Mar 21, 2006 at 11:50:26AM -0800, Emil Cazamir wrote:
 E> I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: 
 E> - physical device:
 E>    |- vlan1 - cbq on vlan1
 E>    |- vlan2 - priq on vlan2
 E>    `- vlan3 - hfsc on vlan3
 
 This may not work, unless you put a bandwidth limit on each vlan interface,
 that is equal to limit of parent interface / number of vlans.
 
 E> At this time i know that altq can make use of only one traffic discipline on an interface, which makes the above case only a wish.
 E> I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.
 
 The first (theoretical) obstacle is that ALTQ is designed to shape traffic on
 an interface, where exists a *bandwidth limit* and a *queue*. These two
 things are property of a physical inteface. The vlan(4) interface doesn't
 have any bandwidth limit. The current implementation has a queue. Packets
 are temporarily queued, before they are sent to the underlying physical
 driver. We have a plan to remove this queue, since it is a performance loss
 for vlan(4) driver.
 
 The second (practical) problem is that after removal of the intermediate
 queue in the vlan(4) driver it will be difficult to add ALTQ support. No
 queueing - no ALTernate Queueing.
 
 -- 
 Totus tuus, Glebius.
 GLEBIUS-RIPN GLEB-RIPE
 I believe that this opinion (ALTQ + vlan is a bad idea) somehow attempts to minimize the benefits which vlan offers, or to make things more complicate than they really are. The reason for which I made the initial request is that I want to treat a vlan interface as an interface and nothing more, making use of some uniform resource configuration technique. Until recently, I used a PC with 3 physical (1xGbE + 2xFE) interfaces to connect a network to two upstream providers. At the same time, I do ip acconting  and traffic shaping with dummynet. Since recently, I can make use of vlans through one of the upstream providers, to reach another subnet under my administration. The main problem is that I'm limited only to use dummynet, and I want to use ALTQ in combination with dummynet to give to my customers more traffic options, while letting them to use more bandwidth to communicate between subnets at a higher speed.
 Here's the big picture:
 provider1 (vlan1)
  `- |             | vlan1-GbE0(myrouter)| lan#1
     |802.1q switch| vlan2-GbE0(myrouter)|
  ,- |             |   --- vlan3 ---     |
 provider2
  `- |another_802.1q_sw|vlan2|another_router | lan#2
     |                 |   --- vlan3 ---     |
 
 service type #1 (85% of all bandwidth): WF2Q+: dummynet with dynamic queues, use weighted fair all available bandwidth until they reach some byte counters.
 service type #2 (15% of all bandwidth): ALTQ and cbq or hfsc with shared bandwidth between few customers with different needs.
 
 My goal is to use dummynet and altq on vlan1, vlan2 and vlan3. Dummynet has the advantage of dynamic queues, which means that i need only one ipfw rule to do the traffic discipline with dummynet. I also have a small number of customers which want to use a service with maximum bandwidth much below to the bandwidth i offer with dummynet, but I can't use dynamic pipes with them because I won't be able to share bandwidth between them.
 The main problem is that I don't have the budget to use a blade server chassis and server modules for this purpose, and I believe that until the number of customers will be below 1000 - 1500, I will be able to use a single computer for this purpose.
 
  
 Best regards,
 Emil Cazamir
 
 
 		
 ---------------------------------
 Yahoo! Messenger with Voice. Make PC-to-Phone Calls to the US (and 30+ countries) for 2/min or less.
 --0-980750068-1143138044=:59095
 Content-Type: text/html; charset=iso-8859-1
 Content-Transfer-Encoding: 8bit
 
 <b style="font-family: courier;"><i>Gleb Smirnoff &lt;glebius@FreeBSD.org&gt;</i></b><span style="font-family: courier;"> wrote:</span><blockquote class="replbq" style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-family: courier;">   Emil,<br><br>On Tue, Mar 21, 2006 at 11:50:26AM -0800, Emil Cazamir wrote:<br>E&gt; I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on h is own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: <br>E&gt; - physical device:<br>E&gt;    |- vlan1 - cbq on vlan1<br>E&gt;    |- vlan2 - priq on vlan2<br
From: Gleb Smirnoff <glebius@FreeBSD.org>
To: Emil Cazamir <enterco@yahoo.com>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/94182: altq support for vlan driver
Date: Mon, 27 Mar 2006 11:11:20 +0400

 On Sun, Mar 26, 2006 at 11:22:54AM -0800, Emil Cazamir wrote:
 E> I see that the only way to work this out is the following:
 E> - I upgrade the system to RELENG-6 or newer (altq has support for bge driver)
 E> - I will attempt to use some undocumented option of pf: packet matching for any ethertype
 E> - I make use of ipfw's packet matching code to enqueue traffic (ipfw can either see "from any to any via vlanX" or at least "from any to any mactype vlan"
 E> - I will make more complicated my configuration scripts
 E> 
 E> Problems which will remain open:
 E> - pf.conf manual page, which doesn't state that how to match packets with ethertype = 802.1Q (it only states that matching is "based on attributes of their layer 3 headers" is outdated. This makes pf integration to FreeBSD to appear as being in beta stage. 
 E> - A simple packet matching and enqueueing rule such as "pass out on vlan1 inet from any to some.ip.add.ress queue vlan1_queue1_out" can be defined only with a stranger form such as: "pass out on vlan1 inet from any to some.ip.add.ress queue bge0_q1" and i see this a little weird. 
 
 I don't see this weird.
 
 E> - For most unexperienced users (and not only for them), an attempt to use a ruleset which looks  OK, and is working fine with physical-only interfaces, can lead to unexpected results. In the following situation: bge0 is the parent of vlan0:
 E> pass out on vlan0 all queue q_egress_1
 E> pass in   on vlan0 all
 E> block on bge0 all
 E> In the above situation a packet which should pass through vlan0 will be blocked by the rule below (not tested, but considering that pf inspects at layer3 address..), and with ipfw this won't happen (neither tested :)). This require a special section on pf.conf man page, to make the users aware of such situations.
 
 I'm not sure, but I think, that the traffic will not be blocked in the above
 ruleset. Please try it and see.
 
 E> - there is no way to make use of different traffic classifiers on a physical interface with more vlan sub-interfaces
 E> - there is only one default queue on a phisical interface, and it is useful to have one default queue per logical interface.
 
 And this is correct. There is no reason to make a queue in a random place in the
 kernel. The queue is made in a place where CPU is waiting for something to complete,
 usually I/O. We can wait for NIC to transmit, or for HDD to write data. What for do
 we wait in vlan(4) driver? Nothing. We enqueue and then dequeue, just wasting CPU
 cycles.
 
 E> Some workarounds for the performance impact you mentioned could be some definition in the kernel configuration file, which should be a trigger for "vlan enhanced performance code", and it's up to FreeBSD's core team whether this should be enabled by default or not. From my point of wiew, this  "vlan enhanced code" should be enabled for those who need it: (multi)GB_ethernet_routers_administrators, long_firewall_ruleset_users, etc.
 
 -- 
 Totus tuus, Glebius.
 GLEBIUS-RIPN GLEB-RIPE
>Unformatted:
 >E&gt;    `- vlan3 - hfsc on vlan3<br><br>This may not work, unless you put a bandwidth limit on each vlan interface,<br>that is equal to limit of parent interface / number of vlans.<br><br>E&gt; At this time i know that altq can 
   make use
   of only one traffic discipline on an interface, which makes the above case only a wish.<br>E&gt; I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagge d frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5. 5-PR ERELEASE as of 2006, march 17.<br><br>The first (theoretical) obstacle is that ALTQ is designed to shape traffic on<br>an interface, where exists a *bandwidth limit* and a *queue*. These two<br>things are property of a physical inteface. The vlan(4)  interface doesn't<br>have any bandwidth limit. The current implementation has a queue. Packets<br>are temporarily queued, before they are sent to the underlying physical<br>driver. We have a plan to remove this queue, since it is
    a
   performance loss<br>for vlan(4) driver.<br><br>The second (practical) problem is that after removal of the intermediate<br>queue in the vlan(4) driver it will be difficult to add ALTQ support. No<br>queueing - no ALTernate Queueing.<br><br>-- <br>Totus  tuus, Glebius.<br>GLEBIUS-RIPN GLEB-RIPE<br></blockquote><span style="font-family: courier;">I believe that this opinion (ALTQ + vlan is a bad idea) somehow attempts to minimize the benefits which vlan offers, or to make things more complicate than they  rea lly are. The reason for which I made the initial request is that I want to treat a vlan interface as an interface and nothing more, making use of some uniform resource configuration technique. Until recently, I used a PC with 3 physical (1xGbE + 2xF E) interfaces to connect a network to two upstream providers. At the same time, I do ip acconting and traffic shaping with dummynet. Since recently, I can make use of vlans through one of the upstream providers, to reach another s
   ubnet
   under my administration. The main problem is that I'm limited only to use dummynet, and I want to use ALTQ in combination with dummynet to give to my customers more traffic options, while letting them to use more bandwidth to communicate between subnet s at a higher speed.</span><br style="font-family: courier;"><span style="font-family: courier;">Here's the big picture:</span><br style="font-family: courier;"><span style="font-family: courier;">provider1 (vlan1)</span><br style="font-family: courier;" ><sp an style="font-family: courier;">&nbsp;`- |&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; | vlan1-GbE0(myrouter)| lan#1</span><br style="font-family: courier;"><span style="font-family: courier;">&nbsp;&nbsp;&nbsp; |802.1q switch| vlan2-G bE0(myrouter)|</span><br style="font-family: courier;"><span style="font-family: courier;">&nbsp;,- |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; --- vlan3 ---&nbsp;&nbsp;&nbsp;&nbsp; |</s
   pan><br
   style="font-family: courier;"><span style="font-family: courier;">provider2<br>&nbsp;`- |another_802.1q_sw|vlan2|another_router | lan#2<br>&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb sp; |&nbsp;&nbsp; --- vlan3 --- &nbsp; &nbsp; |<br><br>service type #1 (85% of all bandwidth): WF2Q+: dummynet with dynamic queues, use weighted fair all available bandwidth until they reach some byte counters.<br>service type #2 (15% of all bandwidth):  ALTQ  and cbq or hfsc with shared bandwidth between few customers with different needs.<br><br>My goal is to use dummynet and altq on vlan1, vlan2 and vlan3. Dummynet has the advantage of dynamic queues, which means that i need only one ipfw rule to do t he traffic discipline with dummynet. I also have a small number of customers which want to use a service with maximum bandwidth much below to the bandwidth i offer with dummynet, but I can't use dynamic pipes with them because I w
   on't be
   able to share bandwidth between them.<br>The main problem is that I don't have the budget to use a blade server chassis and server modules for this purpose, and I believe that until the number of customers will be below 1000 - 1500, I will be able to u se a single computer for this purpose.<br><br> <br>Best regards,<br>Emil Cazamir<br><br></span><p>
  		<hr size=1>Yahoo! Messenger with Voice. <a href="http://us.rd.yahoo.com/mail_us/taglines/postman1/*http://us.rd.yahoo.com/evt=39663/*http://voice.yahoo.com">Make PC-to-Phone Calls</a> to the US (and 30+ countries) for 2/min or less.
  --0-980750068-1143138044=:59095--
