From nobody@FreeBSD.org  Tue Mar  1 15:16:29 2011
Return-Path: <nobody@FreeBSD.org>
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 2C2291065676
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  1 Mar 2011 15:16:29 +0000 (UTC)
	(envelope-from nobody@FreeBSD.org)
Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22])
	by mx1.freebsd.org (Postfix) with ESMTP id 1B4748FC2D
	for <freebsd-gnats-submit@FreeBSD.org>; Tue,  1 Mar 2011 15:16:29 +0000 (UTC)
Received: from red.freebsd.org (localhost [127.0.0.1])
	by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p21FGSnG095247
	for <freebsd-gnats-submit@FreeBSD.org>; Tue, 1 Mar 2011 15:16:28 GMT
	(envelope-from nobody@red.freebsd.org)
Received: (from nobody@localhost)
	by red.freebsd.org (8.14.4/8.14.4/Submit) id p21FGS8p095246;
	Tue, 1 Mar 2011 15:16:28 GMT
	(envelope-from nobody)
Message-Id: <201103011516.p21FGS8p095246@red.freebsd.org>
Date: Tue, 1 Mar 2011 15:16:28 GMT
From: Hans Duedal <hd@onlinecity.dk>
To: freebsd-gnats-submit@FreeBSD.org
Subject: AES-NI breaks OpenSSL client calls 
X-Send-Pr-Version: www-3.1
X-GNATS-Notify:

>Number:         155160
>Category:       kern
>Synopsis:       [aesni] AES-NI breaks OpenSSL client calls
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          closed
>Quarter:        
>Keywords:       
>Date-Required:  
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Mar 01 15:20:07 UTC 2011
>Closed-Date:    Fri Mar 04 06:04:48 UTC 2011
>Last-Modified:  Fri Mar 04 06:04:48 UTC 2011
>Originator:     Hans Duedal
>Release:        8.2
>Organization:
OnlineCity ApS
>Environment:
FreeBSD db3.gw.ocx.dk 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
When cryptodev and aesni(4) are enabled in FreeBSD 8.2, some clients using
OpenSSL can't handshake with SSL servers.

Output of "openssl engine -c -t":
(cryptodev) BSD cryptodev engine
 [RSA, DSA, DH, AES-128-CBC]
     [ available ]
(dynamic) Dynamic engine loading support
     [ unavailable ]

From dmesg:
CPU: Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz (2394.01-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206c2  Family = 6  Model = 2c  Stepping = 2
  Features=0xbfebfbff [shortened]
  Features2=0x29ee3ff [shortened]
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard

I followed this article to enable aes-ni: http://translate.google.com/translate?js=n&prev=_t&ie=UTF-8&layout=2&eotf=1&sl=ru&tl=en&u=http%3A%2F%2Fsysadminblog.ru%2Ffreebsd%2F2011%2F01%2F15%2Ffreebsd-aesni-openssl-openvpn.html&act=url

AES-NI gave a 2x performance boost for 1024 and 8192 byte blocks btw.
>How-To-Repeat:
1. Enable cryptodev and aes_ni by adding the following lines to /boot/loader.conf:
aesni_load="YES"
cryptodev_load="YES"
2. Reboot
3. Connect to an affected ssl host (most hosts excluding google): 
curl -v "https://twitter.com/"
4. Error: "error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record mac"
>Fix:
Disable aes-ni.

>Release-Note:
>Audit-Trail:

From: Hans Duedal <hd@onlinecity.dk>
To: bug-followup@freebsd.org, hd@onlinecity.dk
Cc:  
Subject: Re: kern/155160: [aesni] AES-NI breaks OpenSSL client calls
Date: Wed, 2 Mar 2011 12:18:51 +0100

 --001636832066da7c7c049d7e1365
 Content-Type: text/plain; charset=ISO-8859-1
 
 Does not work:
 curl -v --ciphers AES256-SHA "https://twitter.com/"
 curl -v --ciphers AES256-SHA "https://encrypted.google.com/"
 
 Works:
 curl -v --ciphers AES128-SHA "https://twitter.com/"
 curl -v --ciphers AES128-SHA "https://encrypted.google.com/"
 curl -v --ciphers RC4-SHA "https://twitter.com/"
 curl -v --ciphers CAMELLIA128-SHA "https://oc.nimta.com/"
 curl -v --ciphers CAMELLIA256-SHA "https://oc.nimta.com/"
 
 The problem only affects the AES256 cipher and it's variants
 (DHE-RSA-AES256-SHA & DHE-DSS-AES256-SHA). But openssl s_client still works
 with it:
 openssl s_client -ssl3 -cipher AES256-SHA -state -CAfile
 /usr/local/share/certs/ca-root-nss.crt -connect twitter.com:443
 
 --001636832066da7c7c049d7e1365
 Content-Type: text/html; charset=ISO-8859-1
 Content-Transfer-Encoding: quoted-printable
 
 <div>Does not work:</div><div>curl -v --ciphers AES256-SHA &quot;<a href=3D=
 "https://twitter.com/">https://twitter.com/</a>&quot;</div><div>curl -v --c=
 iphers AES256-SHA &quot;<a href=3D"https://encrypted.google.com/">https://e=
 ncrypted.google.com/</a>&quot;</div>
 <div><br></div><div>Works:</div><div>curl -v --ciphers AES128-SHA &quot;<a =
 href=3D"https://twitter.com/">https://twitter.com/</a>&quot;</div><div>curl=
  -v --ciphers AES128-SHA &quot;<a href=3D"https://encrypted.google.com/">ht=
 tps://encrypted.google.com/</a>&quot;</div>
 <div>curl -v --ciphers RC4-SHA &quot;<a href=3D"https://twitter.com/">https=
 ://twitter.com/</a>&quot;</div><div>curl -v --ciphers CAMELLIA128-SHA &quot=
 ;<a href=3D"https://oc.nimta.com/">https://oc.nimta.com/</a>&quot;</div><di=
 v>
 curl -v --ciphers CAMELLIA256-SHA &quot;<a href=3D"https://oc.nimta.com/">h=
 ttps://oc.nimta.com/</a>&quot;</div><div><br></div><div>The problem only af=
 fects the AES256 cipher and it&#39;s variants (DHE-RSA-AES256-SHA &amp; DHE=
 -DSS-AES256-SHA). But openssl s_client still works with it:</div>
 <div>openssl s_client -ssl3 -cipher AES256-SHA -state -CAfile /usr/local/sh=
 are/certs/ca-root-nss.crt -connect <a href=3D"http://twitter.com:443">twitt=
 er.com:443</a></div>
 
 --001636832066da7c7c049d7e1365--

From: Hans Duedal <hd@onlinecity.dk>
To: bug-followup@FreeBSD.org, Hans Duedal <hd@onlinecity.dk>
Cc:  
Subject: Re: kern/155160: [aesni] AES-NI breaks OpenSSL client calls
Date: Wed, 2 Mar 2011 11:53:32 +0100

 --0016368321b259b945049d7db93e
 Content-Type: text/plain; charset=ISO-8859-1
 
 I should note that the issue does not affect the openssl s_client test
 command.
 
 db3# openssl s_client -quiet -state -CAfile
 /usr/local/share/certs/ca-root-nss.crt -connect twitter.com:443
 SSL_connect:before/connect initialization
 SSL_connect:SSLv2/v3 write client hello A
 SSL_connect:SSLv3 read server hello A
 depth=3 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification
 Authority
 verify return:1
 depth=2 /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006
 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary
 Certification Authority - G5
 verify return:1
 depth=1 /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at
 https://www.verisign.com/rpa (c)06/CN=VeriSign Class 3 Extended Validation
 SSL CA
 verify return:1
 depth=0
 /1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/businessCategory=Private
 Organization/serialNumber=4337446/C=US/postalCode=94107/ST=California/L=San
 Francisco/street=795 Folsom St, Suite 600/O=Twitter, Inc./OU=Twitter
  Operations
 verify return:1
 SSL_connect:SSLv3 read server certificate A
 SSL_connect:SSLv3 read server done A
 SSL_connect:SSLv3 write client key exchange A
 SSL_connect:SSLv3 write change cipher spec A
 SSL_connect:SSLv3 write finished A
 SSL_connect:SSLv3 flush data
 SSL_connect:SSLv3 read finished A
 aaaa
 Status: 500 Internal Server Error
 Content-Type: text/html
 
 <html><body><h1>500 Internal Server Error</h1></body></html>SSL3 alert
 read:warning:close notify
 SSL3 alert write:warning:close notify
 
 Used the ca-root from security/ca_root_nss package to avoid verify issues.
 
 As you can see from my original report, cURL is affected, and so is puppet
 which is ruby based, but I assume that many more clients are affected.
 
 --0016368321b259b945049d7db93e--

From: Patrick Lamaiziere <patfbsd@davenulle.org>
To: Hans Duedal <hd@onlinecity.dk>
Cc: bug-followup@FreeBSD.org <bug-followup@FreeBSD.org>
Subject: Re: kern/155160: [aesni] AES-NI breaks OpenSSL client calls
Date: Wed, 2 Mar 2011 20:45:22 +0100

 Le Wed, 2 Mar 2011 11:20:11 GMT,
 Hans Duedal <hd@onlinecity.dk> a crit :
 
 Hi,
 
 >  The problem only affects the AES256 cipher and it's variants
 
 I think this is fixed by this commit:
 http://svn.freebsd.org/viewvc/base?view=revision&revision=219178
 
 According to a mail from Kostic on stable@ that should affect AES-256
 only.
 
 Regards.
State-Changed-From-To: open->feedback 
State-Changed-By: linimon 
State-Changed-When: Thu Mar 3 00:32:24 UTC 2011 
State-Changed-Why:  
To submitter: did that commit fix your problem? 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155160 

From: Hans Duedal <hd@onlinecity.dk>
To: bug-followup@freebsd.org, hd@onlinecity.dk
Cc:  
Subject: Re: kern/155160: [aesni] AES-NI breaks OpenSSL client calls
Date: Thu, 3 Mar 2011 11:13:17 +0100

 The patch from http://svn.freebsd.org/viewvc/base?view=revision&revision=219178
 / http://www.freebsd.org/cgi/query-pr.cgi?pr=155118 fixes the problem.
 Thank you!
State-Changed-From-To: feedback->closed 
State-Changed-By: linimon 
State-Changed-When: Fri Mar 4 06:04:34 UTC 2011 
State-Changed-Why:  
Apparently fixed by the most recent commit to this file. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=155160 
>Unformatted:
