We are having some difficulties when using Stunnel under load.
The maximum sessions running on a Windows servers seems to be set at 935 concurrent connections.
Our system uses Stunnel to route incoming SSL traffic on port 443 through Openssl to a Java application on port 26789. Both Stunnel and the Java server run on a Windows 2003 server, with between 4GB and 8GB of ram, both running Dual Intel Zeon 2Ghz processors. The problem is that when the Stunnel server is under load, the maximum connections that we can achieve is 935 concurrent connections (Windows threads between 936 and 938), before Stunnel throws errors and starts closing the connections. Here is some additional information about our system:
Stunnel version 4.21 (latest Windows binaries)
OpenSSL 0.9.8g
Stunnel Config:
-----------------------------------------------------------------------
debug = 7
output = stunnel.log
# Use it for client mode
#client = yes
#TIMEOUTidle = 60
socket = a:TCP_NODELAY=1
socket = a:SO_LINGER=a:0
socket = r:TCP_NODELAY=1
socket = r:SO_LINGER=r:0
socket = l:TCP_NODELAY=1
socket = l:SO_LINGER=l:0
# Service-level configuration
[JAVASERVER]
#session = 5000
accept = 443
connect = localhost: 26789
# delay = yes
#transparent = yes
-----------------------------------------------------------------------
We have tried various options, such as the TIMEOUTidle and session parameters for Stunnel, but with no luck. We have even tried running Stunnel separately on a Linux distro (FC4), with even less positive results.
The pertinent errors in the log files are as follows: Connection closed: 178664 bytes sent to SSL, 44646 bytes sent to socket
SSL alert (write): warning: close notify _beginthread: Exec format error (8)
Connection rejected: create_client failed
I can mail full logs of our load tests as well.
Our server will be expected to handle upwards of 10000 connections at a time, we've been researching for about a week now how to get the performance we need, but so far nothing.
Iain
On 2008-02-06, at 19:06, Iain @ Mobile777 wrote:
We have even tried running Stunnel separately on a Linux distro (FC4), with even less positive results.
That's strange. I was testing stunnel with over 10k connections on Linux 2.6.18. You just need to make sure ulimit values (e.g. file descriptors) are high enough. http://stunnel.mirt.net/perf.html
_beginthread: Exec format error (8) Connection rejected: create_client failed
Fascinating. What is the resource utilization (memory, threads, etc.)? It looks like there's a limit of 1000 threads on your Windows 2003 server.
Best regards, Mike (the author of stunnel)
I recently did some work on an app where a problem came up regarding the number of threads on a Windows system... I wasn't deeply involved in that part of the code so don't have all the details. It was caused by the default stack size used by the linker. I believe the default was 1 or 2 MB. So, once enough threads were created to use up all the memory then things started failing. The issue was "fixed"/worked around by lowering the stack sized. I don't know if this is the case here ... but if it is and you want more details then I may still be able to get my hands on some of that code.
-Claus
-----Original Message----- From: stunnel-users-bounces@mirt.net [mailto:stunnel-users-bounces@mirt.net] On Behalf Of Michal Trojnara Sent: Wednesday, February 06, 2008 4:09 PM To: stunnel-users@mirt.net stunnel-users@mirt.net Subject: Re: [stunnel-users] Stunnel Problem - 935 Maximum connections onWindows
On 2008-02-06, at 19:06, Iain @ Mobile777 wrote:
We have even tried running Stunnel separately on a Linux distro (FC4),
with even less positive results.
That's strange. I was testing stunnel with over 10k connections on Linux 2.6.18. You just need to make sure ulimit values (e.g. file descriptors) are high enough. http://stunnel.mirt.net/perf.html
_beginthread: Exec format error (8) Connection rejected: create_client failed
Fascinating. What is the resource utilization (memory, threads, etc.)? It looks like there's a limit of 1000 threads on your Windows 2003 server.
Best regards, Mike (the author of stunnel)
_______________________________________________ stunnel-users mailing list stunnel-users@mirt.net http://stunnel.mirt.net/mailman/listinfo/stunnel-users
Claus Lund wrote:
I recently did some work on an app where a problem came up regarding the number of threads on a Windows system... I wasn't deeply involved in that part of the code so don't have all the details. It was caused by the default stack size used by the linker. I believe the default was 1 or 2 MB. So, once enough threads were created to use up all the memory then things started failing. The issue was "fixed"/worked around by lowering the stack sized. I don't know if this is the case here ... but if it is and you want more details then I may still be able to get my hands on some of that code.
I fixed this issue in stunnel 4.05 (4 years ago).
See the change log for details: http://stunnel.mirt.net/ChangeLog_sdf.html
Best regards, Mike