I am trying to setup stunnel with protocol = proxy through haproxy to my apache2 cluster. I have been using haproxy for a while, and wanted to try this before having to go to nginx, or some other proxy. My config works fine as is if I remove the "accept proxy" from the haproxy.cfg and the protocol=proxy from stunnel config, as soon as those are in, I get a 502 bad gateway.
My haproxy.cfg:
global log 127.0.0.1 local0 log 127.0.0.1 local1 debug #log loghost local0 info maxconn 50000 #chroot /usr/share/haproxy user haproxy group haproxy daemon #debug #quiet
defaults log global mode http option httplog option dontlognull option forwardfor retries 3 option redispatch maxconn 50000 contimeout 5000 clitimeout 60000 srvtimeout 300000 stats uri /haproxy_xxxx stats realm Global\ statistics stats auth xxxxxx:xxxxxxx
frontend dev-stg bind *:80 mode http timeout client 5m option forwardfor default_backend dev-www
frontend dev-stg-ssl bind *:81 accept-proxy mode http option httpclose option forwardfor timeout client 5m default_backend dev-www
backend dev-www mode http timeout connect 10s timeout server 600s balance roundrobin #begin_web_config server ws1 10.0.2.10 maxconn 1000 check port 80 #end_web_config stats uri /haproxy_xxxx stats realm Global\ statistics stats auth xxxxxx:xxxxxxx
My stunnel config:
; ***************************************************************************** ; * Global Options * ; *****************************************************************************
; A copy of some devices and system files is needed within the chroot jail ; Chroot conflicts with configuration file reload and many other features chroot = /var/lib/stunnel4/ ; Chroot jail can be escaped if setuid option is not used setuid = stunnel4 setgid = stunnel4
; PID is created inside the chroot jail pid = /stunnel4.pid
; Debugging stuff (may useful for troubleshooting) debug = 7 output = /var/log/stunnel4/stunnel.log
; ***************************************************************************** ; * Service Defaults (may also be specified in individual service sections) * ; *****************************************************************************
; Certificate/key is needed in server mode and optional in client mode cert = /usr/local/certs/server.crt key = /usr/local/certs/server.key
; Disable support for insecure SSLv2 protocol options = NO_SSLv2 ; Workaround for Eudora bug ;options = DONT_INSERT_EMPTY_FRAGMENTS
; The following options provide additional security at some performance penalty ; Default ECDH/DH parameters are strong/conservative, so it is quite safe to ; comment out these lines in order to get a performance boost ;options = SINGLE_ECDH_USE ;options = SINGLE_DH_USE
; ***************************************************************************** ; * Service Definitions (remove all services for inetd mode) * ; *****************************************************************************
; Example SSL server mode services ; Example SSL front-end to a web server
[proxy] accept = 443 connect = 127.0.0.1:81 ; "TIMEOUTclose = 0" is a workaround for a design flaw in Microsoft SSL ; It does not use SSL close-notify alert designed to prevent truncation attacks TIMEOUTclose = 0 protocol = proxy ; vim:ft=dosini
And here is the syslog for haproxy and stunner (ip address in log and cfgs are changed to protect the innocent ;-) )
Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193127680]: Service [proxy] accepted (FD=3) from 1.2.3.4:35045 Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Service [proxy] started Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG5[1737:140557193119488]: Service [proxy] accepted connection from 1.2.3.4:35045 Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: connect_blocking: connecting 127.0.0.1:81 Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: connect_blocking: s_poll_wait 127.0.0.1:81: waiting 10 seconds Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG5[1737:140557193119488]: connect_blocking: connected 127.0.0.1:81 Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG5[1737:140557193119488]: Service [proxy] connected remote server from 127.0.0.1:49010 Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Remote socket (FD=8) initialized Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: Server-mode proxy protocol negotiations started Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: -> PROXY TCP4 1.2.3.4 10.0.2.9 35045 443 Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: Server-mode proxy protocol negotiations succeeded Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: SSL accepted: new session negotiated Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: Negotiated TLSv1/SSLv3 ciphersuite: DHE-RSA-AES256-SHA (256-bit encryption) Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: Compression: null, expansion: null Jun 7 07:10:50 localhost haproxy[1710]: 1.2.3.4:35045 [07/Jun/2012:07:10:50.496] dev-stg-ssl dev-www/ws1 214/0/1/-1/216 502 754 - - PH-- 0/0/0/0/0 0/0 "GET / HTTP/1.1" Jun 7 07:10:50 localhost haproxy[1710]: 1.2.3.4:35045 [07/Jun/2012:07:10:50.496] dev-stg-ssl dev-www/ws1 214/0/1/-1/216 502 754 - - PH-- 0/0/0/0/0 0/0 "GET / HTTP/1.1" Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Socket closed on read Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Sending close_notify alert Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG6[1737:140557193119488]: SSL_shutdown successfully sent close_notify alert Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG3[1737:140557193119488]: transfer: s_poll_wait: TIMEOUTclose exceeded: closing Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG5[1737:140557193119488]: Connection closed: 204 byte(s) sent to SSL, 137 byte(s) sent to socket Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Remote socket (FD=8) closed Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Local socket (FD=3) closed Jun 7 07:10:50 ec2-204-236-129-168 stunnel: LOG7[1737:140557193119488]: Service [proxy] finished (0 left)
1.2.3.4 = my external ip I am testing with 10.0.2.9 = haproxy/stunnel server 10.0.2.10 = my apache server
Any ideas greatly appreciated.