[stunnel-users] rewrite Destination: when rewriting Host:
Pierre Delaage
delaage.pierre at free.fr
Wed Oct 7 20:32:37 CEST 2009
Hello all on the list,
This mail is a follow up to this old discussion :
http://stunnel.mirt.net/pipermail/stunnel-users/2007-November/001728.html
It talks about Webdav over SSL renaming problem, explanations and
solutions; ie when one setups stunnel between an http client and an
httpS server to access DAV location, some features work (load documents
up and down) but renaming files fails.
Well, after having encountered a similar problem and googled for almost
nothing, I have written a memo on the particular problem of (AND
SOLUTION TO) "MOVE/COPY http methods returning error 502 when one wants
to rename files on a DAV location accessed from a client such as
"Windows WebDAV Miniredir" or other, via stunnel".
To ease googling of this page, all this stuff relates to WebDAV over
SSL, Webdav over Stunnel, CANNOT rename file on a webdav location
accessed via SSL/stunnel on a HTTPS DAV server, and also relates to
"move big file is very long when accessing a DAV location over
SSL/stunnel".Other relevant keywords: "net use over stunnel to dav
location fails in renaming files", "Windows WebDav MiniRedir over
stunnel fails in renaming files".
Anyway I can say that STUNNEL has no bug in this matter, but as being
involved in the particular failing configuration commonly presented in
various online tutorials, I thought that this forum was a good place for
this memo.
I also put this memo in apache bug tracker here :
https://issues.apache.org/bugzilla/show_bug.cgi?id=45449
Let me now include my memo, and apologize for all in the forum if it is
a little bit "boring"...
======================================
I/ The problem:
Briefly: one cannot rename, move or copy (from remote to remote) files
on a https webdav location accessed via stunnel, from dav client such as
Windows Webdav Mini-Redir (ie "net use" mapped drive).
However rename or copy work with "Add Network Place" wizard, but is very
long for big files.
Also, via stunnel, accessing a classic www location works fine (provided
that all url in html files are relative!)
II/ CAUSE:
Quick comments :
there is no bug in Apache,
no bug in stunnel,
and no bug (related to this pb) in Windows webdav miniredir client
so what?...
The method MOVE and COPY of the dav protocol are most often implemented
in dav clients by using an absolute destination url such as
"http://myserver.mydomain/folder1" or
"https://sslserver.otherdomain/folder2", WHILE the "source" document url
is just expressed as a relative path to the "Host" designated in the
http header of the request.
All other http and dav methods, which are essentially some kind of "get
something methods" are most often implemented by using relative path
such "/folderX/adoc" to designate a document of interest, "relative"
means relative to the "host" field defined in the header of the http
request. Note: The PUT method is built the same way as the GET methods:
ie with a relative path to the implicit "document root" of the server
designated in the "Host:" http header field of the request (see RFC 2616
for http, RFC4918 for dav).
Well, from a vocabulary point of view, rfc2616 call my "relative path"
as "abs-path" because they start with a "/", and not a dot (see chapter
3.2.1 of rfc2616).
That said, to secure a dav communication between a XP client and a DAV
server, some people need to set up an ssl (s)tunnel between a WXP/Dav
MiniRedir client and a HTTPS/dav server: this because WXP client are not
able to manage "https connection" directly through "net use". So they
usually use a setup like this on their client machine stunnel.conf file:
accept: localhost:80
connect: remotehost:443
+ some other stuff for cert management.
The XP webdav miniredir client is only able to connect to 80 location.
On XP "net use" does not support https locations nor forcing port number
to any desired value.
So, with such setup, clients are about to build url like this
"http://localhost/remotefolder/remotedoc" in their dav client, then put
this request in an ssl tunnel (stunnel) which will transport this
request to the 443 port of a remote server.
Let's say that the remote server is an Apache one (for example).
A classic configuration of Apache is to setup two ip-based vhosts to
access the same locations (it is very simple and even recommended
because apache does not support ssl access on name-based vhost):
- one ip-based on 80 port (0.0.0.0:80 is the good way to write this,
_default_ and * are not appropriate in that case)
- the other ip-based on 443 port
Some may restrict the 80 vhost to "apache server local network", and
forbid access from the whole internet, and keep 443 access with client
cert validation for external access.
So what happens when MOVE/RENAME is rejected :
the http client will build a MOVE request specifying :
- a source document as a relative "file path", relative to
"localhost", which is an "happily" AMBIGUOUS name (or address if used as
127.0.0.1) that will allow BOTH the client to send its resquest in its
local tunnel-end AND the server to interpret url as relative to ITSELF
(ie the server, not the client) at the other end of the (s)tunnel, so
that requested document can be reaaly reached.
I will explain this further but you have to understand that it is really
a chance that the name "localhost" can serve two different purposes on
two different machines, when it is used an an url exchanged between the
two machines, because an url should express a unic "place" -not two!
I call that the ubiquity of localhost based url, such as
"/localhost/src_doc", relative to the "localhost machine" but in fact
relevant on the SERVER machine
- a destination location as an ABSOLUTE url, such as
"http://localhost/newpath/newdoc_name", also relevant in fact only on
the server machine
Then there are at least two potential sources of pb:
1/ nothing assures the client that the server will be able/allowed to
reach/write to such a location "http://someserver" (even if someserver
is the "itself" remote server contacted by the client, because http urls
are not "the same" as https urls)
2/ even if that location is reachable, the server receives this request
on its 443 port and may expect that url be specified as https, and may
consider receiving an http url as a bug
(because Apache does not want to proxy by default between two of its
vhosts: you have to ask it ...and I think this in NORMAL).
This is where the bug mainly is !
III/ Various comments and questions
1/ the "localhost" ubiquity good bug
-----------------------------------------
There should be a 3rd problem, but this will fortunately not be the case
because of the "ubiquity/ambivalence" of the "locahost" hostname both in
the client network space, and on the server network space:
normally, when the client builds an url beginning with
"http://localhost", it really wants to address and truly adresses a http
request to ITS own host machine:
by the stunnel magic that request comes to the REMOTE server which,
except on particular requests, usually accepts this and serves it :
that means that the server, when seeing such request, interprets
"localhost" as "ITSELF", and as "itself" is in general allowed to access
all its web locations, then the server accepts to serve the request and
send back a document in the tunnel back to the client.
You have to understand that this is NOT (so) normal because the
"locahost" requested by the client is TRULY not the same host machine as
that "understood" by the server.
This is what I call the "localhost" ubiquity good-bug : without this bug
no ssl access through stunnel would work (no www, no dav, no http, no
https, nothing!).
This property is not at all related to a kind of "proxying" role of
stunnel: this is something totally different.
Remember also that stunnel is NOT an http proxy so it transports http
requests "as is".
2/ HTTP + SSL is NOT HTTPS
------------------------------------
Because absolute http url requested by the client are directed to an
https server: as the client does not know that it is talking to an httpS
server, it does not build correct urls.
So it is even a chance that most request works (due to localhost
ambivalence and usage of relative path)...
3/ role of stunnel in HTTP over SSL
-----------------------------------------
In that matter stunnel is not acting as a transparent proxy (this is not
related to the "transparent proxy option" of stunnel) : if it were
perfectly transparent it would not redirect a request to port 80 to a
443 port, and will not "route" requests for THE localhost to a machine
on the internet.
It is also not used as a proxy because the client does not deal with the
remote adress but needs to know the adress of the stunnel client machine
(localhost in the present case).
In fact stunnel, when used only on one side of a end-to-end
communication, is acting as a GATEWAY between two networks : one talking
ssl, the other not.
As such it can translate protocols by removing the payload from an
ordinary tcp/ip packet on one side and putting it in a new ssl tcp/ip
packet on the other side.
We can also say that stunnel acts as a "reverse proxy" hiding a remote
machine behind itself...
Well do not think that tricking your stunnel conf file with "accept 443"
and "connect 443" will solve your problem: because absolute urls in your
requests still begin with http ! and Apache will still not appreciate it
(no implicit proxying).
Moreover MS XP clients will not allow you to mix http / 443 in a net use
command (it is possible on Vista but useless for other reasons: on
vista you can directly connect to https locations with net use)
Stunnel is not either acting as a ...tunnel ! No it does not, because a
pure tunnel will encapsulate requests at one end and remove
encapsulation at the other end.
4/ Some workarounds (but the true solution is later)
-----------------------------------------------------------
WORKAROUND 1 :
if you are impatient to solve your problem, a good solution is to put
stunnel at both ends AND remove SSL support from Apache:
so put stunnel both on the client and on the server to REALLY build a
TRANSPARENT tunnel: I mean transparent to the fact that both ends
believes to talk directly "http on 80 port", as they are in fact
tunneled through an ssl session.
Also note that this true ssl tunnel will work thanks to the "localhost
ambivalence": without that ambivalence, not even a simple get request
would work.
NOTE: please check your apache conf with various directives such as
"usecanonicalname" : I put if off in my conf,
WORKAROUND 2:
of course something else which works is "true https-able client
connecting directly to true https server": this is possible by using the
Windows Vista webdav mini-redir directly talking to apache server on
443. Well in that case you will still have ANOTHER MS BUG : "net use"
DOES not support BOTH "client cert auth" AND "basic auth by
username/password" to access directly a folder on the server (whereas
"add network place" does!). If you want the Vista client to work you
will be obliged to remove "basic auth restriction" at least at the
(dav-)root / location on the server: this is not a good new.
Anyway this will work perfectly if you only use SSL to authenticate the
server, not the client, and keep your basic auth settings unchanged.
Well this solution does not involve stunnel so let us continue...
5/ BUT why other requests such as put/get are working?
--------------------------------------------------------------------
Thanks to the localhost ambivalence AND to relative path in http
requests other than MOVE and COPY (which are dav requests, and which are
the SOLE requests having a destination field, and where this field is
implemented as an absolute URL).
6/ NOTE about absolute url in MOVE/COPY destination field
----------------------------------------------------------------------
Some people think that the RFC4918 does not allow "relative url" in the
destination field : this is WRONG.
But it is a fact that most clients implement "destination" just as
"absolute urls": this is like it is.
Some people may think "absolute url, well ok, so it implies that moving
a doc between two servers or two "services" on the same server should be
possible!" :
yes BUT:
1/ the RFC does not say that it is mandatory ! and
2/ Moreover "miracles" are just not possible if various restrictions
disallow this.
3/ Moreover requesting a "write access" on a location that is not in the
"name space" of the server currently servicing the request is just not
logical, here is why :
if it were the case, it would mean that any server should act as a proxy
to any other : although proxy server exist and are described in the rfc,
a http server is not automatically a proxy server; proxy servers are a
special family of servers that require EXPLICIT configuration, NOT
implicit configuration. So "implicit proxying", although not explicitely
forbidden by the RFC, is a NON SENSE vs the RFC.
Further it must be noted that the proxying mechanisms described in the
RFC2616 are NOT involved when performing HTTP-DAV over SSL ! because
this is a particular case of "proxying at tcp/ip level", not http level,
no the RFC is not relevant to solve this case: ie a server is not
supposed to serve a request in a situation not described in the RFC.
7/ BUT Why "rename is working with "add network place wizard", or some
other dav client:
--------------------------------------------------------------------------------------------------------
Because your dav client implements some tricks :
Some clients (such as Windows "Add Network Place" wizard) first try a
true MOVE or COPY method,
then they receive a 502 error ("Bad gateway") that I really think is a
logical reason for rejecting the request:
the reached host (127.0.0.1:443) is NOT implicitely a PROXY to
127.0.0.1:80 (as implied by the destination url in the MOVE/COPY
request, forged as such by a client believing to act in a http/80
world), so it just CANNOT serve your request.
So when copy/move fail, those clients (as "Add Network Place") try a GET
(to local hard drive) and PUT (to remote machine) : well ok, it seems a
good idea but it is NOT because of network speed (often low in upload)
and delays in case of big files.
IV/ SO what is the solution ?
Remember I already stated two workarounds :
1/ true SSL tunneling by using stunnel BOTH on the client AND the
server, to join a http client/80 and a http/80 server, and removing of
SSL support in Apache:
to secure your server just use apache or firewall restrictions such as
"port 80 only reachable from lan or loopback or localhost!").
If 80 is restricted to localhost, the stunnel server must be installed
on the http server, of course...
2/ On Vista, by directly using "net use https://", forgetting stunnel,
but provided that you disable "basic auth" at least at the root of the
mounted remote location.
But these are just workarounds.
**********
A solution based on stunnel client and apache classic conf directives
exists:
Well you have understood that the problem comes from the fact that
Apache DOES NOT want to act AS A PROXY from "127.0.0.1:443 service" to
"127.0.0.1:80" service.
And I really think it is normal.
So you may have 2 solutions :
1/ by doing some "rewrite" magic in your 443 vhost:
1.1/ either rewrite the destination field as A RELATIVE url
1.2/ either rewrite the destination field as beginning with
https://localhost
BUT ALL THIS WILL NOT be possible using standard apache directives
because apache rewrite engine does not give the ability to rewrite this
field. Should it be able to do that : not sure...and I do not request
the apache team to do that.
2/ the TRUE solution is here :
So you cannot rewrite the request, hmm, BUT you COULD REDIRECT IT to a
service that COULD understand it as it is (because in fact the request
can be considered as well-formed when received by an http server,
although not by an httpS server), by using one of the apache redirection
and/or proxying features.
Well, in that particular case, redirect is NOT the good solution.
The solution is in "PROXYING" !
The PROXYING solution :
When you have finished this paper you will say "all this blah blah" for
a few lines of config...yes but it is necessary to really master apache
conf, in general and because this conf is also able to open big security
holes.
"Forward proxy" is not the solution (you are lucky, I will NOT explain why).
The solution is in "reverse proxying".
See the apache doc about ProxyPass directive, as this is the basis for
setting up reverse proxy...
Anyway, If you roughly configure "ProxyPass" directives, it will work
for MOVE/COPY with stunnel and "net use" but will make FAIL MANY OTHER
things, and in particular for "true https clients talking directly to
the 443 server": because if you have a WELL formed destination url
beginning with HTTPS, a raw proxying will send this to the "80 service"
which will REJECT IT !
SO the solution is to only (reverse) proxy requests that really need to
be rev-proxied !
For that the only solution is to use the " rewritecond...rewriterule
...[P] flag" magics,
to ONLY rev-proxy <<MOVE and COPY methods arriving on 443 vhost AND
having a destination field beginning with "http://">>, which defines
perfectly http requests sent via a client SSL stunnel gateway.
Then here is the code to add this slective reverse proxying in the
apache configuration :
# ================================================================
# IP BASED VIRTUAL HOSTS
# ================================================================
<VirtualHost 0.0.0.0:80>
# ServerName (not used, this is a reminder related to
usecanonicalname-on; we prefer to use usecanonicalname Off!)
ServerName http://LAN-DNS-NAME:80
DocumentRoot "/var/www/html"
</VirtualHost>
<VirtualHost 0.0.0.0:443>
# ServerName useless but it is a reminder (see recommendation in
ServerName apache doc however fort https vhost)
ServerName https://WAN-DNS-NAME:443
DocumentRoot "/var/www/html"
# SSL configuration: PUT WHAT YOU WANT HERE provided that you know what
you are doing !
# -----------------
SSLEngine on
SSLOptions +StrictRequire
# SSL Cipher Suite:
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
# SSL Protocol support:
SSLProtocol all -SSLv2
# I only accept users from my CA
SSLCACertificateFile /etc/pki/tls/cacert.pem
# CRL check
SSLCARevocationFile /etc/pki/tls/cacrl.pem
# Server Certificate Chain File
SSLCertificateChainFile /etc/pki/tls/certs/localhost-chain.crt
# Server Certificate:
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
# Server Private Key:
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
# VERY IMPORTANT
SSLVerifyClient require
SSLVerifyDepth 1
<Location />
SSLRequireSSL
</Location>
# REVERSE PROXYING : solution to "WEBDAV over STUNNEL and RENAME issue",
the solution starts here
#
----------------------------------------------------------------------------------------------------------------------------
RewriteEngine On
# Some env vars useful later
RewriteRule "(.*)" - [env=V443_URL:$1]
RewriteCond %{SSL:SSL_CLIENT_S_DN} "^(.*)$" [NC]
RewriteRule "(.*)" - [env=MY_SSL_CLIENT:%1]
# REVERSE PROXYING : solution to "WEBDAV over STUNNEL and RENAME issue"
#
-----------------------------------------------------------------------------------------
# This is also the solution to problem in renaming files on a subversion
server accessed through dav.
RewriteCond %{REQUEST_METHOD} (MOVE|COPY) [NC]
RewriteCond %{HTTP:Destination} "^(http://.*)$" [NC]
RewriteRule ^/(.*)$ http://127.0.0.1:80/$1 [proxy]
# Note : [proxy] can be abreviated as [P]
# RAF : b flag not supported ? (cf unescape)
# SOME security on the proxy: DO NOT HESITATE TO HARDEN THIS more than
I did
<Proxy http://127.0.0.1:80/>
Order allow,deny
Allow from env=MY_SSL_CLIENT
</Proxy>
</VirtualHost>
# Some useful logs
CustomLog logs/mylogs.log "SSL_DN is %{MY_SSL_CLIENT}e"
CustomLog logs/mylogs.log "V443URL is %{V443_URL}e"
V/ Miscellaneous comments
--------------------------------------
1/ Some people have the same kind of problem:
- when using dav access to subversion location,
- with various dav clients (cadaver, goliath)
- when dealing with error 502 and MOVE/COPY methods
All this problems have the same cause and solution described above :
reverse proxying config on your https server.
2/ a little note about www access (not dav):
Be careful to only have relative url in your various "href" properties
in your links, and "usecanonicalname off" on apache, in order to have
both http and https access working on the same directories.
3/ General note
Think also to have SSLRequireSSL directive in apache conf: do not think
to solve the "move/rename/copy" problem with dav over stunnel by
removing this important directive:
you will only open security holes.
4/ Some people have proposed some similar solutions either by rewriting
or by proxying, but these solutions usually propose to write some kind
of Apache module or filters, and in fact circumvent apache security by
activating a kind of "implicit proxying" which is really a bad idea.
5/ use apache logs (and particularly logs of envvars) and wireshark
(ex-ethereal) to diagnose your dav-client behavior: but remember that
you can only sniff clear traffic SO it in only possible on the client
BUT unfortunately it is NOT possible to sniff the loopback interface on
a XP/Vista machine : the trick is to sniff dav traffic on a clear local
lan without SSL support, to understand how your client manage various
http methods, then to extrapolate when this traffic is transferred
through ssl.
In pure ssl traffic, apache logs can be enriched to track the
"Destination field", with an envvar extracted by the headers through
some rewriterule.
VI/ Some useful links
------------------------.
Good discussion about dav+https (follow the 2 links at the bottom of the
page: one to simple-groupware CMS, the other to "smallvoid" nett use guide)
http://devwfb.blogspot.com/2008/07/how-to-map-https-only-web-folder-to.html
http://www.simple-groupware.de/cms/WebDAV
http://smallvoid.com/article/winnt-webdav-network-drive.html
DAV and SUBVERSION:
http://www.phwinfo.com/forum/alt-apache-configuration/351885-webdav-over-ssl-move-copy.html
http://sethd.org/Site/SubversionProxyFix.html
http://svn.haxx.se/users/archive-2006-03/0544.shtml
Good info about webdav implementation by MS:
http://greenbytes.de/tech/webdav/webdav-redirector-list.html
an howto to setup dav+stunnel
http://www.autohotkey.com/forum/topic44471.html
some other howto:
http://librenix.com/?page=SSL
http://www.howtoforge.com/webdav_with_ssl_and_two_factor_authentication
Apache "bug" that I recommend NOT to fix :
https://issues.apache.org/bugzilla/show_bug.cgi?id=45449
Some stunnel discussion about this:
http://stunnel.mirt.net/pipermail/stunnel-users/2007-November/001728.html
========================
Yours sincerely,
Pierre Delaage
More information about the stunnel-users
mailing list