[stunnel-users] transfer() loop executes not transferring any data and truncated data when using unix sockets

[Michal Trojnara]

Dustin Lundquist wrote:
>> read(7, "HTTP/1.0 200 OKrnContent-Type: text/plainrnContent-Length:
>> 
>> 56391rnrnXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"...,
>> 18432) = 18432
>> poll([{fd=7, events=0}, {fd=3, events=POLLIN|POLLOUT}], 2, 43200000)
>> = 2 ([{fd=7, revents=POLLHUP}, {fd=3, revents=POLLOUT}])

poll() returned POLLHUP for socket 7, so the read side of this socket 
has been closed.

>> read(7, "", 0)                    = 0
>> sendto(6, "<31>Feb  5 15:08:44 stunnel: LOG7[20119:0]: Socket
>> closed on read", 65, MSG_NOSIGNAL, NULL, 0) = 65
>> write(2, "2013.02.05 15:08:44 LOG7[20119:0]: Socket closed on
>> readn", 57) = 57

Stunnel detects is properly.  Looks good.

> Here read() was being called with a size of zero, which according it
> the man page should return 0 and have no other results.

I guess the call to read() is indeed redundant here, as the condition 
could be detected by poll().  On the other hand it should be harmless.

>> > Web client  ---(SSL)--->  stunnel  ---(unix socket)--> haproxy
>> > ---(tcp/http)---> web server

It might be useful to see the configuration file, at lest whether 
"TIMEOUTclose" and "reset" options were specified.

>> > Jan 23 15:24:17 ds420a stunnel: LOG3[22586:139835614443264]:
>> > Compiled/running with OpenSSL 1.0.0-fips 29 Mar 2010

Interesting.  I don't think OpenSSL 1.0.0 is supported by FIPS (neither 
1.2 nor 2.0).

>> > Jan 23 15:24:17 ds420a stunnel: LOG3[22586:139835614443264]:
>> > sock_open_rd=n, sock_open_wr=Y

Read side of the socket was closed.

>> > Jan 23 15:24:17 ds420a stunnel: LOG3[22586:139835614443264]:
>> > SSL_RECEIVED_SHUTDOWN=n, SSL_SENT_SHUTDOWN=Y

close_notify was sent (this SSL functionality is broken in Microsoft 
software).

>> > Jan 23 15:24:17 ds420a stunnel: LOG3[22586:139835614443264]:
>> > sock_can_rd=Y, sock_can_wr=n

Looks like something I need to investigate.  sock_can_rd should=yes 
should not happen with sock_open_rd=no.
I definitely need to investigate how half-closed sockets are handled 
with Unix domain sockets.

>> > input buffer: 0 byte(s), ssl input buffer: 0 byte(s)
[cut]
>> > 2.  Add the "reset=no" parameter to our stunnel.conf (which
>> > seems to fix the problem, but makes me wonder if we're not
>> > opening ourselves up to other problems--  like really
>> > getting stuck in an infinite loop that that watchdog is
>> > intended to avoid.)

I this case (with empty buffers) "reset = no" should be safe.

>> > My best hypothesis at this point is that the transfer() code was
>> > written only with TCP sockets in mind, and that something about
>> > the unix socket in use here is triggering that loop too often.

Unix sockets indeed got much less testing than TCP sockets.

Mike