[stunnel-users] Memory allocated in a different thread at network.c, line 63

[Philippe A.]

Hello Michal,

2012/3/5 Michal Trojnara <Michal.Trojnara at mirt.net>

>
> I am using fork mode because of leaks with pthread. In 4.33+pthread,
>> stunnel virt size (per top) went up to 216m after 40k transactions. In
>> fork mode, stunnel size after the same test was 20744 (kilobytes). In
>> 4.45, I have 187m and 20676k respectively.
>>
>
> I observed the same symptoms. Stunnel grows to ~200MB of virtual memory
> after some time of usage.  Then it stops growing further.  I guess this
> issue is caused by heap fragmentation rather than a memory leak.  Otherwise
> the growth would not be limited to ~200MB.
> https://en.wikipedia.org/wiki/**C_dynamic_memory_allocation<https://en.wikipedia.org/wiki/C_dynamic_memory_allocation>



I ran a test overnight that confirmed your point. I generated 480k+ stunnel
transactions. This morning, stunnel memory usage is virt=187m and res=42m.
My result after 40k transactions was exactly the same (in both cases with
pthread+zlib).



>  I understand that virt includes shared libs, and that any issue with a
>> shared lib will impact virt size. I did not have the opportunity to test
>> without zlib support in openssl. Since we are not using compression, I
>> wonder how much avoiding zlib would help.
>>
>
> zlib only uses lots of memory when it's enabled.
>


I ran another test to clarify this. zlib appears to be adding around 3
megabytes at most. This is at least what I see under my setup and type of
usage.

pthread+zlib, 40k transactions
before: virt=20680, res=792
after: virt=187m, res=42m

pthread+nozlib, 40k transactions
before: virt=20688, res=896
after: virt=185m, res=39m

I have seen no benefit of compiling without zlib in fork mode. I guess it
only makes forking a bit lighter.


2012/3/5 Michal Trojnara <Michal.Trojnara at mirt.net>


> You may also try to set a lower value of MMAP_THRESHOLD to reduce heap
> fragmentation:
> https://www.kernel.org/doc/man-pages/online/pages/man3/malloc.3.html
>


This will be worth a look. Knowing what is influencing stunnel size is nice
since it can avoid us restarting the processes once in a while (which we
currently never do in fork mode).

Thanks for your great help!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stunnel.org/pipermail/stunnel-users/attachments/20120308/abafc791/attachment.html>