From nobody@FreeBSD.org  Tue Feb  1 01:20:34 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id D4D47106566B
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  1 Feb 2011 01:20:34 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id B83758FC0C
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  1 Feb 2011 01:20:34 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p111KY1Q099522
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 1 Feb 2011 01:20:34 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p111KYfE099521;
	Tue, 1 Feb 2011 01:20:34 GMT
	(envelope-from nobody)
Message-Id: <201102010120.p111KYfE099521@red.freebsd.org>
Date: Tue, 1 Feb 2011 01:20:34 GMT
From: Alex <alex@ahhyes.net>
To: freebsd-gnats-submit@FreeBSD.org
Subject: xn0 network interface and PF - Massive performance drop
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         154428
>Category:       kern
>Synopsis:       [xen] xn0 network interface and PF - Massive performance drop
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-xen
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Feb 01 01:30:09 UTC 2011
>Closed-Date:    
>Last-Modified:  Tue Jul 16 04:20:01 UTC 2013
>Originator:     Alex
>Release:        FreeBSD 8.2-RC2
>Organization:
>Environment:
FreeBSD srv.mydomain.net 8.2-RC2 FreeBSD 8.2-RC2 #4: Sun Jan 30 10:15:26 EST 2011     alex@srv.mydomain.net:/usr/obj/usr/src/sys/custom-server  amd64
>Description:
Hi Guys,

Have been forced to file a PR as I have had no answer on this from the freebsd-xen mailing list.

I am running FreeBSD under a XEN HVM environment with a commercial VPS provider. I recently went from running a generic type of kernel to one that includes the XENHVM options. I now have a network interface called xn0 instead of re0, It was obviously necessary to update my pf.conf as the interface name has changed.

All i did was edit the pf.conf file, and replace all instances of re0 with xn0. The performance seems to be aweful. I was wondering why network connectivity was so slow. A download test from apache struggled to do 2KB/s. I disabled pf and suddenly the speed skyrocketed. Any ideas where to look? I have the following in my kernel for PF:

device pf
device pflog
device pfsync
options         ALTQ
options         ALTQ_CBQ        # Class Bases Queuing (CBQ)
options         ALTQ_RED        # Random Early Detection (RED)
options         ALTQ_RIO        # RED In/Out
options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
options         ALTQ_PRIQ       # Priority Queuing (PRIQ)
options         ALTQ_NOPCC      # Required for SMP build

and pf.conf (very basic setup):
--------------------------------

mailblocklist = "{ 69.6.26.0/24 }"
#blacklist = "{ 202.16.0.11 }"

# Rule  0 (xn0)
#pass in quick on xn0 inet proto icmp from any  to (xn0)  label "RULE 0 -- ACCEPT "

#block mail server(s) that continue to try and send me junk
block in quick on xn0 inet proto tcp  from $mailblocklist to (xn0) port 25

#block anyone else who's in the blacklist
#block in quick on xn0 inet from $blacklist to (xn0)

pass in quick on xn0 inet proto tcp  from any  to (xn0) port { 110, 25, 80, 443, 21, 53 } flags any  label "RULE 0 -- ACCEPT "
pass in  quick on xn0 inet proto udp  from any  to (xn0) port 53  label "RULE 0 -- ACCEPT "

#
# Rule  1 (lo0)
pass  quick on lo0 inet  from any  to any no state  label "RULE 1 -- ACCEPT "
#
# Rule  2 (xn0) -- allow all outbound connectivity
pass out  quick on xn0 inet  from any  to any  label "RULE 2 -- ACCEPT "

# Rule  3 (xn0)
# deny all not matched by above
block in quick on xn0 inet  from any  to any no state  label "RULE 3 -- DROP "

--------------------------

Any ideas why I would be seeing such a performance hit? I need to get to the bottom of this as leaving a public facing machine with it's firewall disabled is bad news.

I am not sure whether this a PF or Network interface issue.

>How-To-Repeat:
Install freebsd 8.2RC2 in a XEN HVM environment (could also affect other versions of freebsd), build the XENHVM kernel then enable a simple PF ruleset like above. Test network throughput with PF enabled and also without PF enabled and witness the difference.

>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-xen 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Tue Feb 1 04:53:08 UTC 2011 
Responsible-Changed-Why:  
Over to maintainer(s). 

http://www.freebsd.org/cgi/query-pr.cgi?pr=154428 

From: Alex <joovke@joovke.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/154428: xn0 network interface and PF - Massive performance
 drop
Date: Tue, 01 Feb 2011 14:29:58 +1100

 Confirmed problem still evident in 8.2 RC3 (did a full source csup and 
 buildworld).
 

From: alex <alex@ahhyes.net>
To: <bug-followup@FreeBSD.org>
Cc:  
Subject: Re: kern/154428: [xen] xn0 network interface and PF - Massive
 performance drop
Date: Sun, 13 Feb 2011 14:01:51 +1100

  Hi,
 
  Any update on this? I've had to disable external connections for some 
  services on my VPS due to dictionary/brute force attacks and having no 
  ability to use PF to firewall out the offending IP's/ranges. If nobody 
  is interested, I will go back to a generic kernel.
 
 

From: alex <alex@ahhyes.net>
To: <bug-followup@FreeBSD.org>
Cc:  
Subject: Re: kern/154428: [xen] xn0 network interface and PF - Massive
 performance drop
Date: Sun, 13 Feb 2011 15:21:20 +1100

  Fixed by net.inet.tcp.tso: 1 -> 0
 
  but why?? found this by trial an error.  setting net.inet.tcp.tso to 0 
  with pf enabled gives good performance, if i set it to 1, speeds plumet 
  to below dialup!
 

From: Colin Percival <cperciva@freebsd.org>
To: Alex <joovke@joovke.com>
Cc: freebsd-xen@freebsd.org, bug-followup@FreeBSD.org
Subject: Re: kern/154428: xn0 network interface and PF - Massive performance
 drop
Date: Sat, 12 Feb 2011 20:31:07 -0800

 On 02/12/11 20:18, Alex wrote:
 > Fixed by net.inet.tcp.tso: 1 -> 0
 > 
 > but why?? found this by trial an error.  setting net.inet.tcp.tso to 0
 > with pf enabled gives good performance, if i set it to 1, speeds plumet
 > to below dialup!
 
 There have been problems with Xen and TSO in the past relating to how much
 data gets handed off to the hypervisor at once... why this would cause issues
 only with PF, I have no idea, though.
 
 -- 
 Colin Percival
 Security Officer, FreeBSD | freebsd.org | The power to serve
 Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid

From: alex <alex@ahhyes.net>
To: Cc: bug-followup@FreeBSD.org
Subject: Re: kern/154428: xn0 network interface and PF - Massive performance
 drop
Date: Sun, 13 Feb 2011 15:45:28 +1100

 Beats me. Perhaps someone can look into it as i am out of my league with 
 this one from this point onwards. I am happy to try any patches and 
 rebuild and report the outcomes if need be.
 
 Its a *big* relief to have a firewall again though!
 

From: Alex <joovke@joovke.com>
To: Cc: freebsd-xen@freebsd.org, bug-followup@freebsd.org
Subject: Re: kern/154428: xn0 network interface and PF - Massive performance
 drop
Date: Mon, 14 Feb 2011 23:07:29 +1100

 I stumbled across PR 135178, perhaps there is some relationship with 
 these PR's, though the reporter of that PR has not responded in some time.
 
 On 02/13/11 15:31, Colin Percival wrote:
 > On 02/12/11 20:18, Alex wrote:
 >> Fixed by net.inet.tcp.tso: 1 ->  0
 >>
 >> but why?? found this by trial an error.  setting net.inet.tcp.tso to 0
 >> with pf enabled gives good performance, if i set it to 1, speeds plumet
 >> to below dialup!
 > There have been problems with Xen and TSO in the past relating to how much
 > data gets handed off to the hypervisor at once... why this would cause issues
 > only with PF, I have no idea, though.
 >
 

From: Mark Felder <feld@feld.me>
To: bug-followup@freebsd.org
Cc: alex@ahhyes.net
Subject: Re: kern/154428: [xen] xn0 network interface and PF - Massive
 performance drop
Date: Thu, 23 Aug 2012 16:14:18 -0500

 I've hit this on 9.0-RELEASE as well using XCP 1.5beta as the hypervisor.

From: "Mark Felder" <feld@feld.me>
To: bug-followup@freebsd.org, alex@ahhyes.net
Cc:  
Subject: Re: kern/154428: [xen] xn0 network interface and PF - Massive
 performance drop
Date: Mon, 15 Jul 2013 08:43:30 -0500

 I wasn't able to replicate this on an 8.4 XENHVM kernel -- perhaps this  
 has now been fixed?
 
 When 9.2-RELEASE drops we should test there as well before closing this  
 out.

From: alex@ahhyes.net
To: <bug-followup@freebsd.org>
Cc:  
Subject: Re: kern/154428: [xen] xn0 network interface and PF - Massive
 performance drop
Date: Tue, 16 Jul 2013 14:09:49 +1000

 On 2013-07-15 23:43, Mark Felder wrote:
 > I wasn't able to replicate this on an 8.4 XENHVM kernel -- perhaps
 > this  has now been fixed?
 >
 > When 9.2-RELEASE drops we should test there as well before closing 
 > this  out.
 
 Hi Mark,
 
 You're certain TSO is enabled for the NIC? ie: Not disabled via 
 ifconfig or sysctl?
 
 Cheers,
 Alex.
 
>Unformatted:
