[stunnel-users] rewrite Destination: when rewriting Host:

Pierre Delaage delaage.pierre at free.fr
Wed Oct 7 20:32:37 CEST 2009


Hello all on the list,
This mail is a follow up to this old discussion :
http://stunnel.mirt.net/pipermail/stunnel-users/2007-November/001728.html

It talks about Webdav over SSL renaming problem, explanations and 
solutions; ie when one setups stunnel between an http client and an 
httpS server to access DAV location, some features work (load documents 
up and down) but renaming files fails.

Well, after having encountered a similar problem and googled for almost 
nothing, I have written a memo on the particular problem of (AND 
SOLUTION TO) "MOVE/COPY http methods returning error 502 when one wants 
to rename files on a DAV location accessed from a client such as 
"Windows WebDAV Miniredir" or other, via stunnel".

To ease googling of this page, all this stuff relates to WebDAV over 
SSL, Webdav over Stunnel, CANNOT rename file on a webdav location 
accessed via SSL/stunnel on a HTTPS DAV server, and also relates to 
"move big file is very long when accessing a DAV location over 
SSL/stunnel".Other relevant keywords: "net use over stunnel to dav 
location fails in renaming files", "Windows WebDav MiniRedir over 
stunnel fails in renaming files".

Anyway I can say that STUNNEL has no bug in this matter, but as being 
involved in the particular failing configuration commonly presented in 
various online tutorials, I thought that this forum was a good place for 
this memo.

I also put this memo in apache bug tracker here :
https://issues.apache.org/bugzilla/show_bug.cgi?id=45449

Let me now include my memo, and apologize for all in the forum if it is 
a little bit "boring"...


======================================
I/ The problem:
Briefly: one cannot rename, move or copy (from remote to remote) files 
on a https webdav location accessed via stunnel, from dav client such as 
Windows Webdav Mini-Redir (ie "net use" mapped drive).

However rename or copy work with "Add Network Place" wizard, but is very 
long for big files.
Also, via stunnel, accessing a classic www location works fine (provided 
that all url in html files are relative!)



II/ CAUSE:

Quick comments :
there is no bug in Apache,
no bug in stunnel,
and no bug (related to this pb)  in Windows webdav miniredir client

so what?...

The method MOVE and COPY of the dav protocol are most often implemented 
in dav clients by using an absolute destination url such as 
"http://myserver.mydomain/folder1" or 
"https://sslserver.otherdomain/folder2", WHILE the "source" document url 
is just expressed as a relative path to the "Host" designated in the 
http header of the request.
All other http and dav methods, which are essentially some kind of "get 
something methods" are most often implemented by using relative path 
such "/folderX/adoc" to designate a document of interest, "relative" 
means relative to the "host" field defined in the header of the http 
request. Note:  The PUT method is built the same way as the GET methods: 
ie with a relative path to the implicit "document root" of the server 
designated in the "Host:" http header field of the request (see RFC 2616 
for http, RFC4918 for dav).
Well, from a vocabulary point of view, rfc2616 call my "relative path" 
as "abs-path" because they start with a "/", and not a dot (see chapter 
3.2.1 of rfc2616).

That said, to secure a dav communication between a XP client and a DAV 
server, some people need to set up an ssl (s)tunnel between a WXP/Dav 
MiniRedir client and a HTTPS/dav server: this because WXP client are not 
able to manage "https connection" directly through "net use". So they 
usually use a setup like this on their client machine  stunnel.conf file:
accept: localhost:80
connect: remotehost:443
+ some other stuff for cert management.
The XP webdav miniredir client is only able to connect to 80 location. 
On XP "net use" does not support https locations nor forcing port number 
to any desired value.

So, with such setup, clients are about to build url like this 
"http://localhost/remotefolder/remotedoc" in their dav client, then put 
this request in an ssl tunnel (stunnel) which will transport this 
request to the 443 port of a remote server.
Let's say that the remote server is an Apache one (for example).

A classic configuration of Apache is to setup two ip-based vhosts to 
access the same locations (it is very simple and even recommended 
because apache does not support ssl access on name-based vhost):
- one ip-based on 80 port (0.0.0.0:80 is the good way to write this, 
_default_  and * are not appropriate in that case)
- the other ip-based on 443 port
Some may restrict the 80 vhost to "apache server local network", and 
forbid access from the whole internet, and keep 443 access with client 
cert validation for external access.

So what happens when MOVE/RENAME is rejected :
the http client will build a MOVE request specifying :
   - a source document as a relative "file path", relative to 
"localhost", which is an "happily" AMBIGUOUS name (or address if used as 
127.0.0.1)  that will allow BOTH the client to send its resquest in its 
local tunnel-end AND the server to interpret url as relative to ITSELF 
(ie the server, not the client) at the other end of the (s)tunnel, so 
that requested document can be reaaly reached.
I will explain this further but you have to understand that it is really 
a chance that the name "localhost" can serve two different purposes on 
two different machines, when it is used an an url exchanged between the 
two machines, because an url should express a unic "place" -not two!
I call that the ubiquity of localhost based url, such as 
"/localhost/src_doc", relative to the "localhost machine" but in fact 
relevant on the SERVER machine
   - a destination location as an ABSOLUTE url, such as 
"http://localhost/newpath/newdoc_name", also relevant in fact only on 
the server machine

Then there are at least two potential sources of pb:
1/ nothing assures the client that the server will be able/allowed to 
reach/write to such a location "http://someserver" (even if someserver 
is the "itself" remote server contacted by the client, because http urls 
are not "the same" as https urls)
2/ even if that location is reachable, the server receives this request 
on its 443 port and may expect that url be specified as https, and may 
consider receiving an http url as a bug
(because Apache does not want to proxy by default between two of its 
vhosts: you have to ask it ...and I think this in NORMAL).
This is where the bug mainly is !

III/ Various comments and questions

1/ the "localhost" ubiquity good bug
-----------------------------------------
There should be a 3rd problem, but this will fortunately not be the case 
because of the "ubiquity/ambivalence" of the "locahost" hostname both in 
the client network space, and on the server network space:
normally, when the client builds an url beginning with 
"http://localhost", it really wants to address and truly adresses a http 
request to ITS own host machine:
by the stunnel magic that request comes to the REMOTE server which, 
except on particular requests, usually accepts this and serves it :
that means that the server, when seeing such request,  interprets 
"localhost" as "ITSELF", and as "itself" is in general allowed to access 
all its web locations, then the server accepts to serve the request and 
send back a document in the tunnel back to the client.
You have to understand that this is NOT (so) normal because the 
"locahost" requested by the client is TRULY not the same host machine as 
that "understood" by the server.
This is what I call the "localhost" ubiquity good-bug : without this bug 
no ssl access through stunnel would work (no www, no dav, no http, no 
https, nothing!).
This property is not at all related to a kind of "proxying" role of 
stunnel: this is something totally different.
Remember also that stunnel is NOT an http proxy so it transports http 
requests "as is".

2/ HTTP + SSL is NOT HTTPS
------------------------------------
Because absolute http url requested by the client are directed to an 
https server: as the client does not know that it is talking to an httpS 
server, it does not build correct urls.
So it is even a chance that most request works (due to localhost 
ambivalence and usage of relative path)...

3/ role of stunnel in HTTP over SSL
-----------------------------------------

In that matter stunnel is not acting as a transparent proxy (this is not 
related to the "transparent proxy option" of stunnel) : if it were 
perfectly transparent it would not redirect a request to port 80 to a 
443 port, and will not "route" requests for THE localhost to a machine 
on the internet.
It is also not used as a proxy because the client does not deal with the 
remote adress but needs to know the adress of the stunnel client machine 
(localhost in the present case).

In fact stunnel, when used only on one side of a end-to-end 
communication, is acting as a GATEWAY between two networks : one talking 
ssl, the other not.
As such it can translate protocols by removing the payload from an 
ordinary tcp/ip packet on one side and putting it in a new ssl tcp/ip 
packet on the other side.
We can also say that stunnel acts as a "reverse proxy" hiding a remote 
machine behind itself...

Well do not think that tricking your stunnel conf file with "accept 443" 
and "connect 443" will solve your problem: because absolute urls in your 
requests still begin with http ! and Apache will still not appreciate it 
(no implicit proxying).
Moreover MS XP clients will not allow you to mix http / 443 in a net use 
command  (it is possible on Vista but useless for other reasons: on 
vista you can directly connect to https locations with net use)

Stunnel is not either acting as a ...tunnel ! No it does not, because a 
pure tunnel will encapsulate requests at one end and remove 
encapsulation at the other end.

4/ Some workarounds (but the true solution is later)
-----------------------------------------------------------
WORKAROUND 1 :
if you are impatient  to solve your problem, a good solution is to put 
stunnel at both ends AND remove SSL support from Apache:
so put stunnel both on the client and on the server to REALLY build a 
TRANSPARENT tunnel: I mean transparent to the fact that both ends 
believes to talk directly "http on 80 port", as they are in fact 
tunneled through an ssl session.
Also note that this true ssl tunnel will work thanks to the "localhost 
ambivalence": without that ambivalence, not even a simple get request 
would work.
NOTE: please check your apache conf with various directives such as 
"usecanonicalname" : I put if off in my conf,

WORKAROUND 2:
of course something else which works is "true https-able client 
connecting directly to true https server": this is possible by using the 
Windows Vista webdav mini-redir directly talking to apache server on 
443. Well in that case you will still have ANOTHER MS BUG : "net use" 
DOES not support BOTH "client cert auth" AND "basic auth by 
username/password" to access directly a folder on the server (whereas 
"add network place" does!). If you want the Vista client to work you 
will be obliged to remove "basic auth restriction" at least at the 
(dav-)root / location on the server: this is not a good new.
Anyway this will work perfectly if you only use SSL to authenticate the 
server, not the client, and keep your basic auth settings unchanged.
Well this solution does not involve stunnel so let us continue...


5/ BUT why other requests such as put/get are working?
--------------------------------------------------------------------
Thanks to the localhost ambivalence AND to relative path in http 
requests other than MOVE and COPY (which are dav requests, and which are 
the SOLE requests having a destination field, and where this field is 
implemented as an absolute URL).

6/ NOTE about absolute url in MOVE/COPY destination field
----------------------------------------------------------------------
Some people think that the RFC4918 does not allow "relative url" in the 
destination field : this is WRONG.
But it is a fact that most clients implement "destination" just as 
"absolute urls": this is like it is.

Some people may think  "absolute url, well ok, so it implies that moving 
a doc between two servers or two "services" on the same server should be 
possible!" :
yes BUT:
1/ the RFC does not say that it is mandatory ! and
2/ Moreover "miracles" are just not possible if various restrictions 
disallow this.
3/ Moreover requesting a "write access" on a location that is not in the 
"name space" of the server currently servicing the request is just not 
logical, here is why :
if it were the case, it would mean that any server should act as a proxy 
to any other : although proxy server exist and are described in the rfc, 
a http server is not automatically a proxy server; proxy servers are a 
special family of servers that require EXPLICIT configuration, NOT 
implicit configuration. So "implicit proxying", although not explicitely 
forbidden by the RFC, is  a NON SENSE vs the RFC.
Further it must be noted that the proxying mechanisms described in the 
RFC2616 are NOT involved when performing HTTP-DAV over SSL ! because 
this is a particular case of "proxying at tcp/ip level", not http level, 
no the RFC is not relevant to solve this case: ie a server is not 
supposed to serve a request in a situation not described in the RFC.


7/ BUT Why "rename is working with "add network place wizard", or some 
other dav client:
--------------------------------------------------------------------------------------------------------
Because your dav client implements some tricks :
Some clients (such as Windows "Add Network Place" wizard) first try a 
true MOVE or COPY method,
then they receive a 502 error ("Bad gateway") that I really think is a 
logical reason for rejecting the request:
the reached host (127.0.0.1:443)  is NOT implicitely a PROXY to 
127.0.0.1:80 (as implied by the destination url in the MOVE/COPY 
request, forged as such by a client believing to act in a http/80 
world), so it just CANNOT serve your request.
So when copy/move fail, those clients (as "Add Network Place") try a GET 
(to local hard drive) and PUT (to remote machine) :  well ok, it seems a 
good idea but it is NOT because of network speed (often  low in upload) 
and delays in case of big files.


IV/ SO what is the solution ?

Remember I already stated two workarounds :
1/ true SSL tunneling by using stunnel BOTH on the client AND the 
server, to join a http client/80 and a http/80 server, and removing of 
SSL support in Apache:
to secure your server just use apache or firewall restrictions such as 
"port 80 only reachable from lan or loopback or localhost!").
If 80 is restricted to localhost, the stunnel server must be installed 
on the http server, of course...

2/ On Vista, by directly using "net use https://", forgetting stunnel, 
but provided that you disable "basic auth" at least at the root of the 
mounted remote location.

But these are just workarounds.

**********

A solution based on stunnel client and apache classic conf directives 
exists:

Well you have understood that the problem comes from the fact that 
Apache DOES NOT want to act AS A PROXY from "127.0.0.1:443 service" to 
"127.0.0.1:80" service.
And I really think it is normal.

So you may have 2 solutions :
1/ by doing some "rewrite" magic in your 443 vhost:
1.1/ either rewrite the destination field as A RELATIVE url
1.2/ either rewrite the destination field as beginning with 
https://localhost
BUT ALL THIS WILL NOT be possible using standard apache directives 
because apache rewrite engine does not give the ability to rewrite this 
field. Should it be able to do that : not sure...and I do not request 
the apache team to do that.

2/ the TRUE solution is here :
So you cannot rewrite the request, hmm, BUT you COULD REDIRECT IT to a 
service that COULD understand it as it is (because in fact the request 
can be considered as well-formed when received by an http server, 
although not by an httpS server), by using one of the apache redirection 
and/or proxying features.
Well, in that particular case, redirect is NOT the good solution.
The solution is in "PROXYING" !


The PROXYING solution :
When you have finished this paper you will say "all this blah blah" for 
a few lines of config...yes but it is necessary to really master apache 
conf, in general and because this conf is also able to open big security 
holes.

"Forward proxy" is not the solution (you are lucky, I will NOT explain why).
The solution is in "reverse proxying".
See the apache doc about ProxyPass directive, as this is the basis for 
setting up reverse proxy...

Anyway, If you roughly configure "ProxyPass" directives, it will work 
for MOVE/COPY with stunnel and "net use" but will make FAIL MANY OTHER 
things, and in particular for "true https clients talking directly to 
the 443 server": because  if you have a WELL formed destination url 
beginning with HTTPS, a raw proxying will send this to the "80 service" 
which  will REJECT IT !

SO the solution is to only (reverse) proxy requests that really need to 
be rev-proxied !
For that the only solution is to use the " rewritecond...rewriterule 
...[P] flag" magics,
to ONLY rev-proxy <<MOVE and COPY methods arriving on 443 vhost AND 
having a destination field beginning with "http://">>, which defines 
perfectly http requests sent via a client SSL stunnel gateway.

Then here is the code to add this slective reverse proxying in the 
apache configuration :

# ================================================================
#                       IP BASED VIRTUAL HOSTS
# ================================================================

<VirtualHost 0.0.0.0:80>
# ServerName (not used, this is a reminder related to 
usecanonicalname-on; we prefer to use usecanonicalname Off!)
ServerName http://LAN-DNS-NAME:80
DocumentRoot "/var/www/html"
</VirtualHost>


<VirtualHost 0.0.0.0:443>

# ServerName useless but it is a reminder (see recommendation in 
ServerName apache doc however fort https vhost)
ServerName https://WAN-DNS-NAME:443
DocumentRoot "/var/www/html"

# SSL configuration: PUT WHAT YOU WANT HERE provided that you know what 
you are doing !
# -----------------

SSLEngine on
SSLOptions +StrictRequire

#   SSL Cipher Suite:
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW

#   SSL Protocol support:
SSLProtocol all -SSLv2

# I only accept users from my CA
SSLCACertificateFile /etc/pki/tls/cacert.pem

# CRL check
SSLCARevocationFile /etc/pki/tls/cacrl.pem

# Server Certificate Chain File
SSLCertificateChainFile /etc/pki/tls/certs/localhost-chain.crt

#   Server Certificate:
SSLCertificateFile /etc/pki/tls/certs/localhost.crt

#   Server Private Key:
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

# VERY IMPORTANT

SSLVerifyClient require

SSLVerifyDepth 1

<Location />
SSLRequireSSL
</Location>

# REVERSE PROXYING : solution to "WEBDAV over STUNNEL and RENAME issue", 
the solution starts here
# 
----------------------------------------------------------------------------------------------------------------------------

RewriteEngine On

# Some env vars useful later

RewriteRule "(.*)" - [env=V443_URL:$1]

RewriteCond %{SSL:SSL_CLIENT_S_DN} "^(.*)$" [NC]
RewriteRule "(.*)" - [env=MY_SSL_CLIENT:%1]

# REVERSE PROXYING : solution to "WEBDAV over STUNNEL and RENAME issue"
# 
-----------------------------------------------------------------------------------------
# This is also the solution to problem in renaming files on a subversion 
server accessed through dav.

RewriteCond %{REQUEST_METHOD} (MOVE|COPY) [NC]
RewriteCond %{HTTP:Destination} "^(http://.*)$" [NC]
RewriteRule ^/(.*)$ http://127.0.0.1:80/$1 [proxy]
# Note : [proxy] can be abreviated as [P]
# RAF : b flag not supported ? (cf unescape)

# SOME security on the proxy: DO NOT HESITATE TO HARDEN THIS more  than 
I did
<Proxy http://127.0.0.1:80/>
Order allow,deny
Allow from env=MY_SSL_CLIENT
</Proxy>

</VirtualHost>

# Some useful logs
CustomLog logs/mylogs.log "SSL_DN is  %{MY_SSL_CLIENT}e"
CustomLog logs/mylogs.log "V443URL is  %{V443_URL}e"

V/ Miscellaneous comments
--------------------------------------

1/ Some people have the same kind of problem:
- when using dav access to subversion location,
- with various dav clients (cadaver, goliath)
- when dealing with error 502 and MOVE/COPY methods
All this problems have the same cause and solution described above : 
reverse proxying config on your https server.

2/ a little note about www access (not dav):
Be careful to only have relative url in your various "href" properties 
in your links, and "usecanonicalname off" on apache, in order to have 
both http and https access working on the same directories.

3/ General note
Think also to have SSLRequireSSL directive in apache conf: do not think 
to solve the "move/rename/copy" problem with dav over stunnel by 
removing this important directive:
  you will only open security holes.

4/ Some people have proposed some similar solutions either by rewriting 
or by proxying, but these solutions usually propose to write some kind 
of Apache module or filters, and in fact circumvent apache security by 
activating a kind of "implicit proxying" which is really a bad idea.

5/ use apache logs (and particularly logs of envvars) and wireshark 
(ex-ethereal) to diagnose your dav-client behavior: but remember that 
you can only sniff clear traffic SO it in only possible on the client 
BUT unfortunately it is NOT possible to sniff the loopback interface on 
a XP/Vista machine : the trick is to sniff dav traffic on a clear local 
lan without SSL support, to understand how your client manage various 
http methods, then to extrapolate when this traffic is transferred 
through ssl.
In pure ssl traffic, apache logs can be enriched to track the 
"Destination field", with  an envvar extracted by the headers through 
some rewriterule.


VI/ Some useful links
------------------------.
Good discussion about dav+https (follow the 2 links at the bottom of the 
page: one to simple-groupware CMS, the other to "smallvoid" nett use guide)
http://devwfb.blogspot.com/2008/07/how-to-map-https-only-web-folder-to.html
http://www.simple-groupware.de/cms/WebDAV
http://smallvoid.com/article/winnt-webdav-network-drive.html


DAV and SUBVERSION:
http://www.phwinfo.com/forum/alt-apache-configuration/351885-webdav-over-ssl-move-copy.html
http://sethd.org/Site/SubversionProxyFix.html
  http://svn.haxx.se/users/archive-2006-03/0544.shtml

Good info about webdav implementation by MS:
http://greenbytes.de/tech/webdav/webdav-redirector-list.html

an howto to setup dav+stunnel
http://www.autohotkey.com/forum/topic44471.html

some other howto:
http://librenix.com/?page=SSL
http://www.howtoforge.com/webdav_with_ssl_and_two_factor_authentication

Apache "bug" that I recommend NOT to fix :
https://issues.apache.org/bugzilla/show_bug.cgi?id=45449

Some stunnel discussion about this:
http://stunnel.mirt.net/pipermail/stunnel-users/2007-November/001728.html

========================

Yours sincerely,
Pierre Delaage







More information about the stunnel-users mailing list