From nobody@FreeBSD.org  Tue Mar  3 07:42:45 2009
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 4C30F106564A
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  3 Mar 2009 07:42:45 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21])
	by mx1.freebsd.org (Postfix) with ESMTP id 396128FC1B
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  3 Mar 2009 07:42:45 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from www.freebsd.org (localhost [127.0.0.1])
	by www.freebsd.org (8.14.3/8.14.3) with ESMTP id n237gjhb029617
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 3 Mar 2009 07:42:45 GMT
	(envelope-from nobody@www.freebsd.org)
Received: (from nobody@localhost)
	by www.freebsd.org (8.14.3/8.14.3/Submit) id n237giLo029616;
	Tue, 3 Mar 2009 07:42:44 GMT
	(envelope-from nobody)
Message-Id: <200903030742.n237giLo029616@www.freebsd.org>
Date: Tue, 3 Mar 2009 07:42:44 GMT
From: Vasile Marii <marii.vasile@gmail.com>
To: freebsd-gnats-submit@FreeBSD.org
Subject: poor performance using criptodevice for IPSEC
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         132277
>Category:       kern
>Synopsis:       [crypto] [ipsec] poor performance using cryptodevice for IPSEC
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-net
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Mar 03 07:50:01 UTC 2009
>Closed-Date:    Fri Jan 24 21:27:00 UTC 2014
>Last-Modified:  Fri Jan 24 21:27:00 UTC 2014
>Originator:     Vasile Marii
>Release:        7.01
>Organization:
none
>Environment:
FreeBSD  7.1-RELEASE FreeBSD 7.1-RELEASE #0: Fri Feb 27 11:09:02 EET 2009     root@:/usr/obj/usr/src/sys/IPSEC  i386
>Description:
I'm working to port a cripto accelerating device driver(it's custom made device) from linux (which works fine) to bsd (freebsd 7.1), but i couldn't get the same(decent) results as for linux. The driver for linux and for bsd both started from the corresponding driver for geode LX cripto accelerator. I concluded that it's not the device and the bottleneck is somewhere in the kernel(interface between Network stack and Cripto Framework). I modified the original glxsb(geode crypto accelerator) driver and made it return immediately after receiving a cripto task (so the device actually does nothing aka device is taking zero time to cript any block of data) and the data is actually not cripted. I made this for debugging purposes to see if the kernel delivers enough data to the device. The netperf results between the two exactly the same machines(with a tunnel(AES-CBC with HMAC_SHA256) between them) with the exactly the same(modified original)driver shows a throughput of maximum 20Mbp
 s(without IPSEC tunnel i can get 94,1 Mbps so the network is ok).
I've seen similar problems on some threads regarding VIA(which should work with 1,1 Gbps throughput).
I've tested the device not cripting network traffic (meaning "feed" the device manually and give it data immediately after it finishes the previous) and i can get a full speed of 117 Mbps(meaning it should be enough for my needs for 100Mbps NIC).
Does anybody have any better results on glxsb or via?(i mean a netperf test between two machines) or there is a hack or a setting in the kernel or somewhere else?

Thanks!
>How-To-Repeat:
use glxsb driver for IPSEC. Using userspace testing shows good results on this driver though.
>Fix:


>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: freebsd-bugs->freebsd-net 
Responsible-Changed-By: linimon 
Responsible-Changed-When: Tue Mar 3 14:06:44 UTC 2009 
Responsible-Changed-Why:  
Over to -net on the theory that the problem might be there. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=132277 

From: "Vasile Marii" <marii.vasile@gmail.com>
To: <bug-followup@FreeBSD.org>,
	<marii.vasile@gmail.com>
Cc:  
Subject: Re: kern/132277: [crypto] [ipsec] poor performance using cryptodevice for IPSEC
Date: Wed, 4 Mar 2009 11:41:47 +0200

 This is a multipart message in MIME format.
 
 ------=_NextPart_000_0001_01C99CBE.34A8B240
 Content-Type: text/plain;
 	charset="us-ascii"
 Content-Transfer-Encoding: 7bit
 
 http://forum.pfsense.org/index.php/topic,14581.msg77206.html#msg77206
 
 
 ------=_NextPart_000_0001_01C99CBE.34A8B240
 Content-Type: text/html;
 	charset="us-ascii"
 Content-Transfer-Encoding: quoted-printable
 
 <html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
 xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
 xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
 xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
 xmlns=3D"http://www.w3.org/TR/REC-html40">
 
 <head>
 <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
 charset=3Dus-ascii">
 <meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)">
 <style>
 <!--
  /* Font Definitions */
  @font-face
 	{font-family:Calibri;
 	panose-1:2 15 5 2 2 2 4 3 2 4;}
  /* Style Definitions */
  p.MsoNormal, li.MsoNormal, div.MsoNormal
 	{margin:0cm;
 	margin-bottom:.0001pt;
 	font-size:11.0pt;
 	font-family:"Calibri","sans-serif";}
 a:link, span.MsoHyperlink
 	{mso-style-priority:99;
 	color:blue;
 	text-decoration:underline;}
 a:visited, span.MsoHyperlinkFollowed
 	{mso-style-priority:99;
 	color:purple;
 	text-decoration:underline;}
 span.EmailStyle17
 	{mso-style-type:personal-compose;
 	font-family:"Calibri","sans-serif";
 	color:windowtext;}
 .MsoChpDefault
 	{mso-style-type:export-only;}
 @page Section1
 	{size:612.0pt 792.0pt;
 	margin:70.85pt 70.85pt 70.85pt 70.85pt;}
 div.Section1
 	{page:Section1;}
 -->
 </style>
 <!--[if gte mso 9]><xml>
  <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
 </xml><![endif]--><!--[if gte mso 9]><xml>
  <o:shapelayout v:ext=3D"edit">
   <o:idmap v:ext=3D"edit" data=3D"1" />
  </o:shapelayout></xml><![endif]-->
 </head>
 
 <body lang=3DEN-US link=3Dblue vlink=3Dpurple>
 
 <div class=3DSection1>
 
 <p =
 class=3DMsoNormal>http://forum.pfsense.org/index.php/topic,14581.msg77206=
 .html#msg77206<o:p></o:p></p>
 
 </div>
 
 </body>
 
 </html>
 
 ------=_NextPart_000_0001_01C99CBE.34A8B240--
 

From: Patrick =?ISO-8859-15?Q?Lamaizi=E8re?= <patfbsd@davenulle.org>
To: bug-followup@FreeBSD.org
Cc: Vasile Marii <marii.vasile@gmail.com>
Subject: Re: misc/132277: poor performance using criptodevice for IPSEC
Date: Sun, 8 Mar 2009 20:56:01 +0100

 Le Tue, 3 Mar 2009 07:42:44 GMT,
 Vasile Marii <marii.vasile@gmail.com>:
 
 Hi,
 
 > Does anybody have any better results on glxsb or via?(i mean a netperf
 > test between two machines) or there is a hack or a setting in the
 > kernel or somewhere else?
 
 I've made some tests on IPsec with glxsb and the performances are very
 bad (around 14 Mbits).
 
 I think the problem is that glxsb handles only one request at a time.
 When it is busy, it blocks the Open Crypto Framework with ERESTART and
 it unblocks the OCF when the previous request is completed. Then the OCF
 has to wake up and to resubmit the request. It looks like this performs
 very badly when using it with IPsec.
 
 If glxsb processes the requests synchronously it performs quite better,
 around 50 Mbits.
 
 I will look for glxsb.

From: Patrick =?ISO-8859-15?Q?Lamaizi=E8re?= <patfbsd@davenulle.org>
To: bug-followup@FreeBSD.org
Cc: Vasile Marii <marii.vasile@gmail.com>
Subject: Re: misc/132277: poor performance using criptodevice for IPSEC
Date: Sat, 14 Mar 2009 13:05:52 +0100

 Le Sun, 8 Mar 2009 20:00:11 GMT,
 Patrick Lamaizire <patfbsd@davenulle.org>:
 
 >  I've made some tests on IPsec with glxsb and the performances are
 > very bad (around 14 Mbits).
 >  
 >  I think the problem is that glxsb handles only one request at a time.
 >  When it is busy, it blocks the Open Crypto Framework with ERESTART
 > and it unblocks the OCF when the previous request is completed. Then
 > the OCF has to wake up and to resubmit the request. It looks like
 > this performs very badly when using it with IPsec.
 >  
 >  If glxsb processes the requests synchronously it performs quite
 > better, around 50 Mbits.
 
 I've filled a PR with a patch for glxb(4): kern/132622
 
 Regards.

From: Vasile Marii <marii.vasile@gmail.com>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/132277: [crypto] [ipsec] poor performance using cryptodevice 
	for IPSEC
Date: Mon, 16 Mar 2009 11:26:46 +0200

 Thanks Patrick.
 Anyway, that patch solves the problem for glxsb but let's admit that
 there is still a problem with the OCF interface or OCF itself because
 on linux geode performs much better: around 90 Mbps.
 I must insist that there is a problem with the OCF or something
 because it's not ok to get half a speed on a 100Mbps NIC only because
 interaction with IP stack(as i think) or interface with OCF is not ok.
 
 
 --
 Vasile Marii
State-Changed-From-To: open->closed 
State-Changed-By: jmg 
State-Changed-When: Fri Jan 24 21:26:35 UTC 2014 
State-Changed-Why:  
closed because kern/132622 has a patch... 

http://www.freebsd.org/cgi/query-pr.cgi?pr=132277 
>Unformatted:
