Hi,
I am trying to set up syslog + stunnel in a large environment. I am curious about the experience of members of this mailing list regarding how stunnel + syslog-ng scale.
I set up a test environment using stunnel 3.26 (because that's what is in my debian installation)[*]. I configured stunnel to run as a daemon (starting on boot), and syslog passes off messages and receives messages from localhost:514. In the stunnel log, it tells me that there is a limit of 500 clients, and it seems that with stunnel 3.x, it must be recompiled to increase this limit. I found some posts on this list that say that while stunnel 3.x uses select(), stunnel 4.x uses poll(), which is much more efficient. So I figure that if I will have to roll my own package, I may as well upgrade to 4.x at the same time. Agree? If so, which version?
It's my understanding that this configuration will create a persistent connection between the client and server, holding it open until such time as syslog needs to send a message across it. How many clients have you experienced being able to connect to the log aggregator? My logs are rather sparse, so I expect I will hit a limit based on processor / filehandle / memory usage before I start overloading the local disk. Eventually, I realize that I will have to build a tree structure with intermediate nodes aggregating logs and passing them on to the central host, but I would like to know where people have hit that limit. I would love to have ~5000 clients connected to each aggregating server. Is this within the realm of experience?
Does anybody have tuning suggestions for such high numbers of connections? I saw one person mention on the mailing list that compiling without libwrap allowed him to pass ~2500 connections (though he didn't give a new ceiling).
Thanks,
-ben
[*] I was actually impressed at how easy this was. Aside from having to write my own /etc/init.d/ scripts to start the client and server, I could bring down either end of the stunnel connection, and things would just pick up where they left off when the tunnel was reconnected. Add monit into the picture and you've got a nice resilient secure logging system. Slick!
addendum that I forgot. My debian does actually have stunnel 4.09, I just havn't gotten around to testing it yet (is there a compelling reason to upgrade to 4.20 from 4.09?). And I'm running debian on linux 2.6.16-2-686-smp. ;)
-ben
On Thu, Oct 18, 2007 at 03:00:56PM -0700, Ben Hartshorne wrote:
Hi,
I am trying to set up syslog + stunnel in a large environment. I am curious about the experience of members of this mailing list regarding how stunnel + syslog-ng scale.
I set up a test environment using stunnel 3.26 (because that's what is in my debian installation)[*]. I configured stunnel to run as a daemon (starting on boot), and syslog passes off messages and receives messages from localhost:514. In the stunnel log, it tells me that there is a limit of 500 clients, and it seems that with stunnel 3.x, it must be recompiled to increase this limit. I found some posts on this list that say that while stunnel 3.x uses select(), stunnel 4.x uses poll(), which is much more efficient. So I figure that if I will have to roll my own package, I may as well upgrade to 4.x at the same time. Agree? If so, which version?
It's my understanding that this configuration will create a persistent connection between the client and server, holding it open until such time as syslog needs to send a message across it. How many clients have you experienced being able to connect to the log aggregator? My logs are rather sparse, so I expect I will hit a limit based on processor / filehandle / memory usage before I start overloading the local disk. Eventually, I realize that I will have to build a tree structure with intermediate nodes aggregating logs and passing them on to the central host, but I would like to know where people have hit that limit. I would love to have ~5000 clients connected to each aggregating server. Is this within the realm of experience?
Does anybody have tuning suggestions for such high numbers of connections? I saw one person mention on the mailing list that compiling without libwrap allowed him to pass ~2500 connections (though he didn't give a new ceiling).
Thanks,
-ben
[*] I was actually impressed at how easy this was. Aside from having to write my own /etc/init.d/ scripts to start the client and server, I could bring down either end of the stunnel connection, and things would just pick up where they left off when the tunnel was reconnected. Add monit into the picture and you've got a nice resilient secure logging system. Slick!
-- Ben Hartshorne email: ben@hartshorne.net http://ben.hartshorne.net
stunnel-users mailing list stunnel-users@mirt.net http://stunnel.mirt.net/mailman/listinfo/stunnel-users
On Thu, Oct 18, 2007 at 03:33:28PM -0700, Ben Hartshorne wrote:
addendum that I forgot. My debian does actually have stunnel 4.09, I just havn't gotten around to testing it yet
In case you do decide to upgrade, AFAIK the package in lenny/sid ought to compile in sarge without problem.
On Thu, Oct 18, 2007 at 03:00:56PM -0700, Ben Hartshorne wrote:
I set up a test environment using stunnel 3.26 (because that's what is in my debian installation)
Stunnel 4.x *is* available in Debian, in the aptly named package stunnel4 :)
[*] I was actually impressed at how easy this was. Aside from having to write my own /etc/init.d/ scripts to start the client and server
Do you mean scripts to start/stop stunnel? If so, could you tell me how the Debian scripts could be improved to support your use?
On Thu, Oct 18, 2007 at 06:35:13PM -0500, Luis Rodrigo Gallardo Cruz wrote:
On Thu, Oct 18, 2007 at 03:00:56PM -0700, Ben Hartshorne wrote:
[*] I was actually impressed at how easy this was. Aside from having to write my own /etc/init.d/ scripts to start the client and server
Do you mean scripts to start/stop stunnel? If so, could you tell me how the Debian scripts could be improved to support your use?
The stunnel4 package has start/stop scripts, but the stunnel 3 package doesn't. This actually makes sense, because rather than using a config file, you specify the arguments on the command line, which isn't particularly well suited to a start script. I wrote two, stunnel-client and stunnel-server (since you might want to run both to aggregate connections) and put all the relevant variables (destination hostnames, port numbers, etc.) in /etc/default/.
I can send them to you off-list if you like, but I don't imagine the stunnel 3 packages will be maintained much longer, so it might not be worth the effort.
-ben
On Fri, Oct 19, 2007 at 07:33:26AM -0700, Ben Hartshorne wrote:
The stunnel4 package has start/stop scripts, but the stunnel 3 package doesn't.
Oh, that's it. Sorry, I just recently took over them.
I can send them to you off-list if you like, but I don't imagine the stunnel 3 packages will be maintained much longer, so it might not be worth the effort.
Indeed not worth it. Thanks anyways.
-
Ben Hartshorne -
Luis Rodrigo Gallardo Cruz