<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EstiloDeEmail19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EstiloDeEmail20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EstiloDeEmail21
{mso-style-type:personal;
font-family:"Calibri",sans-serif;}
span.EstiloDeEmail23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:708843498;
mso-list-type:hybrid;
mso-list-template-ids:1145714854 1763576842 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\.\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:1710689715;
mso-list-template-ids:2140936590;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=PT-BR link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>Sorry I replay but I saw that it�s a different thread that I opened.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>Forget the email bellow.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b>De:</b> Luis Monteiro <luis.monteiro440@gmail.com> <br><b>Enviada em:</b> sexta-feira, 18 de janeiro de 2019 22:27<br><b>Para:</b> 'Eric Eberhard' <flash@vicsmba.com>; 'Tamar Pedersen' <tamar.pedersen@bbraunusa.com>; stunnel-users@stunnel.org<br><b>Assunto:</b> RES: [stunnel-users] How can stunnel use openssl HW cryptodev encryption<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>Eric is not being used in a corporate network. I use BPS/VE to create traffic for several tests including HTTPS and the Ixia implementation drain my resources very fast even in a Xeon Gold 6140 dual. I�m talking about create from 500Mbps to 10 or 20Gbps of HTTPS without have to buy (Money that I do not have) a expensive boxlike PS1 (Perfect Storm 1) from Ixia. Pretty expensive so.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>I�m using the Log feature of iptables putting a lot of marks to see how the traffic is going and coming.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>First Probrem. Even with Source transparent mode without a nat prerouting nat destination to TLS_Client (6.0.0.1 and the clients create by BPS goes from 6.0.0.2 and ahead) it didn�t. BPS works much better if I do not have to say that there is a proxy in the path. In the manual of Stunnel it says that support both source/destination transparent but I couldn’t setup it.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>If someone has a clue that could help I will appreciate.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>Luis<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b>De:</b> stunnel-users <<a href="mailto:stunnel-users-bounces@stunnel.org">stunnel-users-bounces@stunnel.org</a>> <b>Em nome de </b>Eric Eberhard<br><b>Enviada em:</b> sexta-feira, 18 de janeiro de 2019 21:23<br><b>Para:</b> 'Tamar Pedersen' <<a href="mailto:tamar.pedersen@bbraunusa.com">tamar.pedersen@bbraunusa.com</a>>; <a href="mailto:stunnel-users@stunnel.org">stunnel-users@stunnel.org</a><br><b>Assunto:</b> Re: [stunnel-users] How can stunnel use openssl HW cryptodev encryption<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>I’ll give you two pieces of advice that almost everyone on the list won’t agree with </span><span lang=EN-US style='font-family:Wingdings;color:#1F497D'>J</span><span lang=EN-US style='color:#1F497D'><br><br>1) Make a static openssl and static stunnel and locate them someplace apart from the standard locations (/usr/local/company_name/lib is what I use). This means if anyone messes with openssl or stunnel you won’t be affected – and it will always work as it is static – and part of your application – the user does not even need your libraries. I have a personal distaste for dynamic linking … mostly because I have a lot of customers that update a lot of things (including openssl – a lot – and stunnel sometimes) … and then wonder why things stop working. A 10 year old openssl and stunnel – all static – will still run and work fine past all updates and user messing around. I choose when I want to update openssl and stunnel (meaning I look to see if there is something new I need or want). As a result I missed the keep alive and poodle bugs – I did not update until after both were fixed.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>2) Forget hardware implementation – geez – modern computers are so darn fast that I cannot imagine you really need that level of “speed up” versus the grief you are handling. I have customers that exchange millions (4+) XML documents a day, all through stunnel, all through inetd (also not efficient supposedly – just reliable and always works and needs no management) – and have no problems. I am using IBM p Series (AIX) and these machines even at the low level are fast … but I also use some SCO and Linux and certainly with lesser volume they are fine as well.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>3) OK – 3 is really – use inetd, so much easier and always works (assuming you have Unix). If inetd crashes Unix crashes so … see number 2 for reasons </span><span lang=EN-US style='font-family:Wingdings;color:#1F497D'>J</span><span lang=EN-US style='color:#1F497D'><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Of course, these ideas won’t help much if you don’t have a Unix variation or if you are really that tight on performance (although if you are I’d suggest hardware upgrades!).<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'>Good luck with your project,<br><br>Eric<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US>From:</span></b><span lang=EN-US> stunnel-users [<a href="mailto:stunnel-users-bounces@stunnel.org">mailto:stunnel-users-bounces@stunnel.org</a>] <b>On Behalf Of </b>Tamar Pedersen<br><b>Sent:</b> Wednesday, January 16, 2019 1:06 PM<br><b>To:</b> <a href="mailto:stunnel-users@stunnel.org">stunnel-users@stunnel.org</a><br><b>Subject:</b> [stunnel-users] How can stunnel use openssl HW cryptodev encryption<o:p></o:p></span></p></div></div><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Hello,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I am evaluating stunnel, to see if it is a viable solution for providing encryption in a system that contains an Atmel processor which includes a HW accelerated encryption block. I am just ramping up on stunnel, and figured I should capture what I have done so far. My questions will come towards the end of my email. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>My research indicates that stunnel incorporates openssl. I have been able to use openssl independently, to access the cryptodev HW encryption engine, in the Linux kernel module located in /lib/modules/4.14.79/extra/cryptodev.ko. When openssl is run without accessing the cryptodev engine (cryptodev module not loaded), I get the pure SW encryption implementation provided by default in openssl. When I run bench mark speed tests using openssl, using SW encryption, I see the following results:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># time -v openssl speed -evp aes-128-cbc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 16 size blocks: 1689887 aes-128-cbc's in <span style='color:red'>2.95s<o:p></o:p></span></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 64 size blocks: 568389 aes-128-cbc's in <span style='color:red'>2.95s</span><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 256 size blocks: 151550 aes-128-cbc's in <span style='color:red'>2.96s</span><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 1024 size blocks: 38599 aes-128-cbc's in <span style='color:red'>2.96s</span><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 8192 size blocks: 4845 aes-128-cbc's in <span style='color:red'>2.95s<o:p></o:p></span></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>OpenSSL 1.0.2p-fips 14 Aug 2018<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>built on: reproducible build, date unspecified<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>compiler: arm-laird-linux-gnueabi-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -I/home/sii/wb50n_space2_legacy_6.0.0.x/wb/buildroot/output/wb50n_space2_legacy/host/arm-buildroot-linux-gnueabi/sysroot/usr/local/ssl/fips-2.0/include -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>The 'numbers' are in 1000s of bytes per second processed.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>aes-128-cbc 9165.49k 12331.15k 13107.03k 13353.17k 13454.32k<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Command being timed: "openssl speed -evp aes-128-cbc"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> User time (seconds): 14.81<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> System time (seconds): 0.10<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Percent of CPU this job got: 99%<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 15.06s<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average shared text size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average unshared data size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average stack size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average total size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Maximum resident set size (kbytes): 13376<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average resident set size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Major (requiring I/O) page faults: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Minor (reclaiming a frame) page faults: 145<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New";color:red'> Voluntary context switches: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New";color:red'> Involuntary context switches: 721<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Swaps: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> File system inputs: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> File system outputs: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Socket messages sent: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Socket messages received: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Signals delivered: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Page size (bytes): 4096<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Exit status: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>When I load the cryptodev module, and take advantage of the accelerated hardware encryption the benchmark tests are significantly faster. Here is what those results look like.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># modprobe cryptodev<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># time -v openssl speed -evp aes-128-cbc<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 16 size blocks: 44163 aes-128-cbc's in <span style='color:red'>0.12s<o:p></o:p></span></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 64 size blocks: 31345 aes-128-cbc's in <span style='color:red'>0.15s</span><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 256 size blocks: 18923 aes-128-cbc's in <span style='color:red'>0.11s</span><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 1024 size blocks: 13847 aes-128-cbc's in <span style='color:red'>0.13s</span><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>Doing aes-128-cbc for 3s on 8192 size blocks: 8427 aes-128-cbc's in <span style='color:red'>0.06s<o:p></o:p></span></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>OpenSSL 1.0.2p-fips 14 Aug 2018<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>built on: reproducible build, date unspecified<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>compiler: arm-laird-linux-gnueabi-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -I/home/sii/wb50n_space2_legacy_6.0.0.x/wb/buildroot/output/wb50n_space2_legacy/host/arm-buildroot-linux-gnueabi/sysroot/usr/local/ssl/fips-2.0/include -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>The 'numbers' are in 1000s of bytes per second processed.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>aes-128-cbc 5888.40k 13373.87k 44038.98k 109071.75k 1150566.40k<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Command being timed: "openssl speed -evp aes-128-cbc"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> User time (seconds): 0.59<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> System time (seconds): 8.72<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Percent of CPU this job got: 61%<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Elapsed (wall clock) time (h:mm:ss or m:ss): 0m 15.11s<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average shared text size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average unshared data size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average stack size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average total size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Maximum resident set size (kbytes): 13792<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Average resident set size (kbytes): 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Major (requiring I/O) page faults: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Minor (reclaiming a frame) page faults: 144<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New";color:red'> Voluntary context switches: 41154<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New";color:red'> Involuntary context switches: 3321<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Swaps: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> File system inputs: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> File system outputs: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Socket messages sent: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Socket messages received: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Signals delivered: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Page size (bytes): 4096<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> Exit status: 0<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>As can be seen in the results (hi-lighted in red), the average speed to do aes-128-cbc encryption jumped from around 2.95 s to 0.10 s. Also of interest is the context switches are significantly higher when running hardware encryption, because of interrupts and overhead to use the hardware engine. I can also look at /proc/interrupts and see significant increases in atmel-aes interrupt counts when using the cryptodev HW acceleration encryption engine. This gives a good indication that the cryptodev module is in use, and is doing encryption.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I would like to try to figure out how to allow stunnel to take advantage of the cryptodev HW acceleration encryption engine available in openssl. I have made some attempts, but so far, I have not been able to determine if stunnel is successfully using the cryptodev engine. Here is what I have done with stunnel. I already have a client and server successfully communicating with each other using stunnel. To verify this I used the “nc” utility to send characters back and forth between two different machines. The stunnel.conf file, on the server, is out of the box. I’m interested in encrypting on the client side. Here is my current client.conf file, in /etc/stunnel:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># cat client.conf<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>debug = 7<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>output = /tmp/stunnel-server.log<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>pid = /tmp/stunnel.pid<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>engine = cryptodev<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[test]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>verify = 1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>client = yes<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>accept = 127.0.0.1:2000<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>connect = 192.168.0.220:30000<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>CAfile = /etc/stunnel/certificate.crt<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>engineNum = 1<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I am attempting to set up the cryptodev to be the configured engine for the client. I am able to start stunnel, using client.conf, as follows:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># stunnel /etc/stunnel/client.conf<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>If I do a “ps” command to display processes, I can see that stunnel is running in the background. At this point, I can use “nc” to send data, as follows:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># nc 127.0.0.1 2000 < /tmp/long_file.txt<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I am able to see the text from long_file.txt on the server, which is also running nc. The problem is that I don’t see interrupts increasing in /proc/interrupts, which leaves me wondering if I have not configured stunnel correctly to use the cryptodev engine. If I try to remove the cryptodev module as this point, while stunnel is running, I receive a message that it is in use, as follows:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># modprobe -r cryptodev<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>modprobe: FATAL: Module cryptodev is in use.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>If I kill the stunnel process, I am able to successfully remove the cryptodev module, which seems to suggest stunnel is the process using the cryptodev module. Also, once I have removed the cryptodev module, I can’t restart stunnel. Instead, I get the following errors back:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># stunnel /etc/stunnel/client.conf<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[.] stunnel 5.44 on arm-buildroot-linux-gnueabi platform<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[.] Compiled/running with OpenSSL 1.0.2p-fips 14 Aug 2018<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[.] Threading:FORK Sockets:POLL,IPv6 TLS:ENGINE,FIPS,OCSP,PSK,SNI<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[ ] errno: (*__errno_location ())<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[.] Reading configuration from file /etc/stunnel/client.conf<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[.] UTF-8 byte order mark not detected<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[ ] Enabling support for engine "cryptodev"<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[!] error queue: 2606A074: error:2606A074:engine routines:ENGINE_by_id:no such engine<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[!] error queue: 260B6084: error:260B6084:engine routines:DYNAMIC_LOAD:dso not found<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[!] error queue: 25070067: error:25070067:DSO support routines:DSO_load:could not load the shared library<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[!] ENGINE_by_id: 25066067: error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[!] /etc/stunnel/client.conf:5: "engine = cryptodev": Failed to open the engine<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Again, this suggests stunnel is trying to use cryptodev. I just don’t know how to prove I am taking advantage of the HW encryption acceleration engine. I never see interrupts updating in /proc/interrupts when using nc, while stunnel is running. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>So, here are my questions:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><ol style='margin-top:0cm' start=1 type=1><li class=MsoListParagraph style='margin-left:0cm;mso-list:l0 level1 lfo3'><span lang=EN-US>Does it look like I have things set up correctly in client.conf, to use the cryptodev engine?<o:p></o:p></span></li><li class=MsoListParagraph style='margin-left:0cm;mso-list:l0 level1 lfo3'><span lang=EN-US>If client.conf is correct, how can I prove that stunnel is using the cryptodev engine, since I don’t see the expected interrupts?<o:p></o:p></span></li></ol><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>One idea is that the cryptodev module might not support the type of encryption being requested by the certificate, so openssl falls back to the pure SW encryption implementation. I know the Atmel chip in question supports the following:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'># openssl engine –t –c<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>(cryptodev) cryptodev engine<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>[RSA, DSA, DH, DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC, MD5, SHA1, SHA256, SHA384, SHA512]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> [ available ]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>(dynamic) Dynamic engine loading support<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'> [ unavailable ]<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'>#<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I was able to decode the contents of the certificate, and it says it is sha256WithRSAEncryption. My engine supports SHA256 and RSA, but does it support combining, like SHA256WithRSA? I’m not sure. I’ll keep chasing that one.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thanks for any guidance on how to use the cryptodev in stunnel.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Regards,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Tamar<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Courier New"'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p></div></body></html>