<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:708843498;
mso-list-type:hybrid;
mso-list-template-ids:1145714854 1763576842 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\.\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">After several days of digging around on the web, without success, I finally got something working. I’m not sure if anyone else is having the same problem with hardware acceleration working in stunnel, since
I couldn’t seem to find much on the topic. However, if I can save anyone the trouble I have been through, I figured I should put it out there as a public service.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I still don’t have all the answers, but here is what worked for me.
</span><span style="color:#1F497D">In my testing, I put together two configuration files, one for a client, client.conf, and one for a server, server.conf. Here they are:</span><span style="font-family:"Courier New";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"># cat /etc/stunnel/client.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">debug = 7<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">output = /tmp/stunnel-client.log<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">fips = no<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">pid = /var/run/stunnel_client.pid<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">[my_config]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">verify = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">client = yes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">accept = 127.0.0.1:2000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">connect = 192.168.0.127:30000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">CAfile = /etc/stunnel/certificate.crt<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"># cat /etc/stunnel/server.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">debug = 7<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">outut = /tmp/stunnel-server.log<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">cert = /etc/stunnel/stunnel.pem<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">CAFile = /etc/stunnel/certificate.crt<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">ciphers = ECDHE-RSA-AES256-SHA384<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">options = NO_SSLv2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">options = NO_SSLv3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">fips = no<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">engine = auto<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">pid = /var/run/stunnel_server.pid<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">[test]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">verify = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">accept = 30000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">connect = 127.0.0.1:60000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The thing that seemed to be the kicker was the line that specified the ciphers in server.conf. I didn’t have a line specifying a cipher in my original server.conf file. I noticed in my stunnel-server.log file
that the negotiated cipher suite was ECDHE-RSA-AES256-GCM-SHA384. I started some investigation to try to figure out where this came from. I still don’t know why this cipher was selected, but I have made some observations. I was able to get a list of ciphers
supported by openssl, by typing the following command:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"># openssl ciphers –v<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">In general, there appear to be some ciphers that are supported by the hardware encryption engine, and others that are not. If the hardware encryption engine does not support the cipher, then it looks like you
get the pure software encryption provided by openssl. In my case, the hardware acceleration engine is on an Atmel chip, and is accessed by the cryptodev engine. The default negotiated cipher I was getting, ECDHE-RSA-AES256-GCM-SHA384, doesn’t seem to be
supported by my hardware accelerator. However, if you take GCM out of the name, you get ECDHE-RSA-AES256-SHA384, which does seem to be supported in my system. I imagine the ciphers supported by any particular chip may vary. When I use this cipher, I start
to see the atmel-aes interrupts incrementing, as I expected in /proc/interrupts. I believe this verifies the hardware acceleration engine is being used. I’m calling this success, but I am still left with some head scratchers.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I also set engine = auto, in my server.conf file. If I do this, I see lines in the stunnel-server.log that show:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Enabling automatic engine support<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Engine #1 (cryptodev) registered<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Engine #2 (dynamic) registered<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Automatic engine support enabled<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">If I set engine = cryptodev, then I don’t see these lines, but it still seems to work. Instead I get a lines that say:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Enabling support for engine “cryptodev”<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">UI set for engine #1 (cryptodev)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Initializing engine #1 (cryptodev)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D">Engine #1 (cryptodev) initialized<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The hardware acceleration in cryptodev seems to work whether engine is set to cryptodev or auto. Not sure what the difference is. Any experts know the difference?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I’m also trying to figure out how to know what ciphers I should expect to be supported on my hardware. If I type the command, openssl engine –t –c, I see the following:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A"># openssl engine –t –c<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A">(cryptodev) cryptodev engine<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A">[RSA, DSA, DH, DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC, MD5, SHA1, SHA256, SHA384, SHA512]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A"> [ available ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A">(dynamic) Dynamic engine loading support<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A"> [ unavailable ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:#44546A"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The cipher that works with cryptodev has a name that contains some of the items in this list, but not all. This is not a list of ciphers, but it does contain strings that show up in supported cipher names.
For instance, GCM is not in the list. The cipher, with GCM in the name, defaulted to running software encryption. ECDHE, is not in the list, but the cipher that worked on hardware had this in the name. How do I know which ciphers should work, without just
going manually through the list of all ciphers and trying each one, to see if it runs on my hardware, or not?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">In summary, I got something to work, but I still have some work to do, to understand how everything plays together.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Tamar</span><span style="font-family:"Courier New";color:#44546A"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Tamar Pedersen <br>
<b>Sent:</b> Monday, January 21, 2019 8:56 AM<br>
<b>To:</b> Eric Eberhard <flash@vicsmba.com>; stunnel-users@stunnel.org<br>
<b>Subject:</b> RE: [stunnel-users] How can stunnel use openssl HW cryptodev encryption<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks for the response. I was wondering if anyone was out there
</span><span style="font-family:Wingdings;color:#1F497D">J</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">In the end, I may need to do encryption without taking advantage of hardware acceleration, but I was hoping to do some benchmark tests with and without hardware acceleration to see what the difference in measured
performance would actually be. Our project is in an embedded system, with a small ARM core running Linux, which has a lot of other jobs to do beyone encrypting data. The CPU utilization looked like it was at 99% when we ran pure software encryption using
openssl stand alone. We obviously are not running full throttle all of the time, so we may still be able to keep up. I also still need to characterize the block size of data that needs to be transmitted. The hardware acceleration doesn’t buy much for small
block sizes. In fact, because of all of the system overhead on context switches, it can actually be slower than pure software encryption for small block sizes. However as block sizes increase, there is an inflection point, where hardware acceleration really
starts to kick in. In the end, the value of hardware encryption will depend on block sizes, which will depend on customer needs that have not yet been well defined, but I’d love to have some headroom if we end up requiring large block sizes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I still have not given up on the idea of trying to get hardware encryption working. I know openssl has the capability to use the hardware encryption in the Atmel chip, when I use it alone. I just need to figure
out the right hooks to get stunnel to configure openssl correctly. I’d love to hear from anyone who is using hardware encryption in stunnel, even if you only send the configuration file you are using. Just looking for some examples. I have only seen one
example of a cryptography engine in stunnel, in the default stunnel.conf file, but it was for a Microsoft CryptoAPI engine. I’d love to see someone who used a Linux based cryptodev device. Has anyone out there tried this?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Tamar<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Eric Eberhard [<a href="mailto:flash@vicsmba.com">mailto:flash@vicsmba.com</a>]
<br>
<b>Sent:</b> Friday, January 18, 2019 6:23 PM<br>
<b>To:</b> Tamar Pedersen <<a href="mailto:tamar.pedersen@bbraunusa.com">tamar.pedersen@bbraunusa.com</a>>;
<a href="mailto:stunnel-users@stunnel.org">stunnel-users@stunnel.org</a><br>
<b>Subject:</b> RE: [stunnel-users] How can stunnel use openssl HW cryptodev encryption<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I’ll give you two pieces of advice that almost everyone on the list won’t agree with
</span><span style="font-family:Wingdings;color:#1F497D">J</span><span style="color:#1F497D"><br>
<br>
1) Make a static openssl and static stunnel and locate them someplace apart from the standard locations (/usr/local/company_name/lib is what I use). This means if anyone messes with openssl or stunnel you won’t be affected – and it will always work as it is
static – and part of your application – the user does not even need your libraries. I have a personal distaste for dynamic linking … mostly because I have a lot of customers that update a lot of things (including openssl – a lot – and stunnel sometimes) …
and then wonder why things stop working. A 10 year old openssl and stunnel – all static – will still run and work fine past all updates and user messing around. I choose when I want to update openssl and stunnel (meaning I look to see if there is something
new I need or want). As a result I missed the keep alive and poodle bugs – I did not update until after both were fixed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">2) Forget hardware implementation – geez – modern computers are so darn fast that I cannot imagine you really need that level of “speed up” versus the grief you are handling. I have customers that exchange millions
(4+) XML documents a day, all through stunnel, all through inetd (also not efficient supposedly – just reliable and always works and needs no management) – and have no problems. I am using IBM p Series (AIX) and these machines even at the low level are fast
… but I also use some SCO and Linux and certainly with lesser volume they are fine as well.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">3) OK – 3 is really – use inetd, so much easier and always works (assuming you have Unix). If inetd crashes Unix crashes so … see number 2 for reasons
</span><span style="font-family:Wingdings;color:#1F497D">J</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Of course, these ideas won’t help much if you don’t have a Unix variation or if you are really that tight on performance (although if you are I’d suggest hardware upgrades!).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Good luck with your project,<br>
<br>
Eric<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> stunnel-users [<a href="mailto:stunnel-users-bounces@stunnel.org">mailto:stunnel-users-bounces@stunnel.org</a>]
<b>On Behalf Of </b>Tamar Pedersen<br>
<b>Sent:</b> Wednesday, January 16, 2019 1:06 PM<br>
<b>To:</b> <a href="mailto:stunnel-users@stunnel.org">stunnel-users@stunnel.org</a><br>
<b>Subject:</b> [stunnel-users] How can stunnel use openssl HW cryptodev encryption<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal">I am evaluating stunnel, to see if it is a viable solution for providing encryption in a system that contains an Atmel processor which includes a HW accelerated encryption block. I am just ramping up on stunnel, and figured I should capture
what I have done so far. My questions will come towards the end of my email. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My research indicates that stunnel incorporates openssl. I have been able to use openssl independently, to access the cryptodev HW encryption engine, in the Linux kernel module located in /lib/modules/4.14.79/extra/cryptodev.ko. When
openssl is run without accessing the cryptodev engine (cryptodev module not loaded), I get the pure SW encryption implementation provided by default in openssl. When I run bench mark speed tests using openssl, using SW encryption, I see the following results:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># time -v openssl speed -evp aes-128-cbc<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 16 size blocks: 1689887 aes-128-cbc's in
<span style="color:red">2.95s<o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 64 size blocks: 568389 aes-128-cbc's in
<span style="color:red">2.95s</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 256 size blocks: 151550 aes-128-cbc's in
<span style="color:red">2.96s</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 1024 size blocks: 38599 aes-128-cbc's in
<span style="color:red">2.96s</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 8192 size blocks: 4845 aes-128-cbc's in
<span style="color:red">2.95s<o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">OpenSSL 1.0.2p-fips 14 Aug 2018<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">built on: reproducible build, date unspecified<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">compiler: arm-laird-linux-gnueabi-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -O3 -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -I/home/sii/wb50n_space2_legacy_6.0.0.x/wb/buildroot/output/wb50n_space2_legacy/host/arm-buildroot-linux-gnueabi/sysroot/usr/local/ssl/fips-2.0/include
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">The 'numbers' are in 1000s of bytes per second processed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">aes-128-cbc 9165.49k 12331.15k 13107.03k 13353.17k 13454.32k<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Command being timed: "openssl speed -evp aes-128-cbc"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> User time (seconds): 14.81<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> System time (seconds): 0.10<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Percent of CPU this job got: 99%<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 15.06s<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average shared text size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average unshared data size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average stack size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average total size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Maximum resident set size (kbytes): 13376<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average resident set size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Major (requiring I/O) page faults: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Minor (reclaiming a frame) page faults: 145<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:red"> Voluntary context switches: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:red"> Involuntary context switches: 721<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Swaps: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> File system inputs: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> File system outputs: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Socket messages sent: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Socket messages received: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Signals delivered: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Page size (bytes): 4096<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Exit status: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">When I load the cryptodev module, and take advantage of the accelerated hardware encryption the benchmark tests are significantly faster. Here is what those results look like.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># modprobe cryptodev<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># time -v openssl speed -evp aes-128-cbc<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 16 size blocks: 44163 aes-128-cbc's in
<span style="color:red">0.12s<o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 64 size blocks: 31345 aes-128-cbc's in
<span style="color:red">0.15s</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 256 size blocks: 18923 aes-128-cbc's in
<span style="color:red">0.11s</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 1024 size blocks: 13847 aes-128-cbc's in
<span style="color:red">0.13s</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Doing aes-128-cbc for 3s on 8192 size blocks: 8427 aes-128-cbc's in
<span style="color:red">0.06s<o:p></o:p></span></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">OpenSSL 1.0.2p-fips 14 Aug 2018<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">built on: reproducible build, date unspecified<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">compiler: arm-laird-linux-gnueabi-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -O3 -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -I/home/sii/wb50n_space2_legacy_6.0.0.x/wb/buildroot/output/wb50n_space2_legacy/host/arm-buildroot-linux-gnueabi/sysroot/usr/local/ssl/fips-2.0/include
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">The 'numbers' are in 1000s of bytes per second processed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">aes-128-cbc 5888.40k 13373.87k 44038.98k 109071.75k 1150566.40k<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Command being timed: "openssl speed -evp aes-128-cbc"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> User time (seconds): 0.59<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> System time (seconds): 8.72<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Percent of CPU this job got: 61%<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 15.11s<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average shared text size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average unshared data size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average stack size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average total size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Maximum resident set size (kbytes): 13792<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Average resident set size (kbytes): 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Major (requiring I/O) page faults: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Minor (reclaiming a frame) page faults: 144<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:red"> Voluntary context switches: 41154<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New";color:red"> Involuntary context switches: 3321<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Swaps: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> File system inputs: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> File system outputs: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Socket messages sent: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Socket messages received: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Signals delivered: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Page size (bytes): 4096<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> Exit status: 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">As can be seen in the results (hi-lighted in red), the average speed to do aes-128-cbc encryption jumped from around 2.95 s to 0.10 s. Also of interest is the context switches are significantly higher when running hardware encryption,
because of interrupts and overhead to use the hardware engine. I can also look at /proc/interrupts and see significant increases in atmel-aes interrupt counts when using the cryptodev HW acceleration encryption engine. This gives a good indication that the
cryptodev module is in use, and is doing encryption.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I would like to try to figure out how to allow stunnel to take advantage of the cryptodev HW acceleration encryption engine available in openssl. I have made some attempts, but so far, I have not been able to determine if stunnel is successfully
using the cryptodev engine. Here is what I have done with stunnel. I already have a client and server successfully communicating with each other using stunnel. To verify this I used the “nc” utility to send characters back and forth between two different
machines. The stunnel.conf file, on the server, is out of the box. I’m interested in encrypting on the client side. Here is my current client.conf file, in /etc/stunnel:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># cat client.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">debug = 7<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">output = /tmp/stunnel-server.log<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">pid = /tmp/stunnel.pid<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">engine = cryptodev<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[test]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">verify = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">client = yes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">accept = 127.0.0.1:2000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">connect = 192.168.0.220:30000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">CAfile = /etc/stunnel/certificate.crt<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">engineNum = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am attempting to set up the cryptodev to be the configured engine for the client. I am able to start stunnel, using client.conf, as follows:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># stunnel /etc/stunnel/client.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">If I do a “ps” command to display processes, I can see that stunnel is running in the background. At this point, I can use “nc” to send data, as follows:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># nc 127.0.0.1 2000 < /tmp/long_file.txt<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">I am able to see the text from long_file.txt on the server, which is also running nc. The problem is that I don’t see interrupts increasing in /proc/interrupts, which leaves me wondering if I have not configured stunnel correctly to use
the cryptodev engine. If I try to remove the cryptodev module as this point, while stunnel is running, I receive a message that it is in use, as follows:<o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># modprobe -r cryptodev<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">modprobe: FATAL: Module cryptodev is in use.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If I kill the stunnel process, I am able to successfully remove the cryptodev module, which seems to suggest stunnel is the process using the cryptodev module. Also, once I have removed the cryptodev module, I can’t restart stunnel. Instead,
I get the following errors back:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># stunnel /etc/stunnel/client.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[.] stunnel 5.44 on arm-buildroot-linux-gnueabi platform<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[.] Compiled/running with OpenSSL 1.0.2p-fips 14 Aug 2018<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[.] Threading:FORK Sockets:POLL,IPv6 TLS:ENGINE,FIPS,OCSP,PSK,SNI<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[ ] errno: (*__errno_location ())<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[.] Reading configuration from file /etc/stunnel/client.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[.] UTF-8 byte order mark not detected<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[ ] Enabling support for engine "cryptodev"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[!] error queue: 2606A074: error:2606A074:engine routines:ENGINE_by_id:no such engine<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[!] error queue: 260B6084: error:260B6084:engine routines:DYNAMIC_LOAD:dso not found<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[!] error queue: 25070067: error:25070067:DSO support routines:DSO_load:could not load the shared library<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[!] ENGINE_by_id: 25066067: error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[!] /etc/stunnel/client.conf:5: "engine = cryptodev": Failed to open the engine<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">Again, this suggests stunnel is trying to use cryptodev. I just don’t know how to prove I am taking advantage of the HW encryption acceleration engine. I never see interrupts updating in /proc/interrupts when using nc, while stunnel is
running. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So, here are my questions:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">1.)<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Does it look like I have things set up correctly in client.conf, to use the cryptodev engine?<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">2.)<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>If client.conf is correct, how can I prove that stunnel is using the cryptodev engine, since I don’t see the expected interrupts?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">One idea is that the cryptodev module might not support the type of encryption being requested by the certificate, so openssl falls back to the pure SW encryption implementation. I know the Atmel chip in question supports the following:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""># openssl engine –t –c<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">(cryptodev) cryptodev engine<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[RSA, DSA, DH, DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC, MD5, SHA1, SHA256, SHA384, SHA512]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> [ available ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">(dynamic) Dynamic engine loading support<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""> [ unavailable ]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">I was able to decode the contents of the certificate, and it says it is sha256WithRSAEncryption. My engine supports SHA256 and RSA, but does it support combining, like SHA256WithRSA? I’m not sure. I’ll keep chasing that one.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for any guidance on how to use the cryptodev in stunnel.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal">Tamar<o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>