vADC Blog

Stingray Traffic Manager Forward Proxy and Tunneling SSL (06/21/2008)

by markbod on ‎06-25-2012 10:21 AM (5,767 Views)

Stingray Traffic Manager Forward Proxy and Tunneling SSL (06/21/2008)

In September 2007, Riverbed released Stingray Traffic Manager version 4.2. One of the new features in the 4.2 release was 'Forward Proxy Mode', which allowed Stingray Traffic Manager to be used as a forward proxy.

A little while after the release, Ben from Stingray's Development Team kindly provided a community article giving step by step instructions on how to configure Stingray Traffic Manager as a Web Cache Proxy Server. Ben's post concentrated on proxying HTTP requests, however proxy servers can sometimes be configured to proxy HTTPS requests too.

This article will provide a TrafficScript™ example that can be used to proxy those HTTPS requests

The CONNECT Method

So why doesn't the original TrafficScript also work with HTTPS? Well, when a browser attempts to connect to a resource over HTTP, it simply sends the request to the proxy and waits for a response; it's all HTTP.

However, when a browser wants to retrieve a secure page over HTTPS, the browser sends a CONNECT request instead. If it used HTTP, then the connection between the browser and the proxy would be in clear text, which is obviously bad, plus the proxy server would have to contain a full SSL stack to handle the connection. Not a problem for Stingray, but not all proxy servers have our SSL stack :-)

The internet gods decided that an all around simpler solution would be to just ask the proxy to open a TCP socket and then let the browser and the remote server handle all the SSL communication themselves. In a nutshell: the HTTPS proxy uses a special CONNECT method to locate the server, and then streams the SSL data directly between the endpoints. It's not HTTP and the HTTP virtual server used for other proxy requests won't work here.

97_23080.png

The Solution

So what if you need to proxy both HTTP and HTTPS traffic? You need two Virtual Servers.

One should be configured as described in Ben's article, and this will handle the HTTP traffic. The second Virtual Server should be configured using the "Generic Client First" protocol and this will handle HTTPS. Once you have the virtual server configured, you'll need to add a TrafficScript request rule.....

  1. # Check for SSL connect requests. If it's not a CONNECT then break
  2. $request = request.getLine();
  3. if( ! string.regexMatch($request,"CONNECT (.*?)Smiley Sad[0-9]*?) HTTP.*" ) )
  4.    break;
  5. # This is an SSL request
  6. $host = $1;
  7. $port = $2;
  8. # If the port is not 443 then disallow the connection
  9. if ( $port != 443 ) {
  10.    request.sendResponse( "HTTP/1.0 403 Forbidden\r\n\r\n" );
  11. } else {
  12.    # Resolve the hostname to an IP address
  13.    $ip = net.dns.resolveHost( $host );
  14.    if ( $ip == "" ) {
  15.       request.sendResponse("HTTP/1.0 404 Unknown Host\r\n\r\n");
  16.    }
  17.    # Select the destination node, and tell the client we're connected.
  18.    pool.select("Forward Proxy Pool", $ip, 443);
  19.    request.sendResponse("HTTP/1.0 200 Connection Established\r\n\r\n");
  20. }

In the code above we first check to see if this request is actually a proxy CONNECT request, if it isn't then we break out of the rule. If we are a CONNECT request then we need to extract the hostname and the port provided in the request. I have decided to only allow requests going to port 443, and send back a "403 Forbidden" response code if they're going anywhere else. The RFC for SSL Tunneling says this:

...the proxy cannot verify that the protocol being spoken is really SSL, and so the proxy configuration should explicitly limit allowed connections to well-known SSL ports (such as 443 for HTTPS, 563 for SNEWS, as assigned by the Internet Assigned Numbers Authority).

Of course Stingray Traffic Manager can allow clients to connect to any port if you want to allow it, but please bear those words in mind should you decide to remove the port restriction.

The next job for our TrafficScript is to resolve the hostname to an IP address (this part shamelessly stolen from Ben). If we can't resolve the host we return a "404 Unknown Host" error.

The last two lines are quite interesting, you'll note we are using pool.select() instead of pool.use(). This is because the Stingray Traffic Manager needs to respond directly to the client's CONNECT request. If we used pool.use() TrafficScript processing would stop and the CONNECT line would be sent to the remote HTTPS server, which is not what we want. Instead we select the pool for use with the data stream and send back a "200 Connection Established" response to the client. The SSL handshake will follow and we will stream all of the data to and from the remote server unhindered.

Using only one Virtual Server

You could do all this using only one virtual server of course. HTTP is a client first protocol, and will run just fine over the virtual server we're using for HTTPS.

However, you will have to re-write Ben's code using the low level request.get() and request.set() functions because the http classes will not be available. Once you've done that and merged it with my code, you'll discover that you don't have access to Content-Caching, Compression, or any other HTTP-specific features, which, I'm sure you'll agree, is less than ideal.

There is an alternative. Remember my code exits if the request does not use the "CONNECT" method, so the HTTPS proxy can be configured to use a Ben's HTTP proxy Virtual Server as it's default pool. Once the TrafficScript identifies that the current request is not for HTTPS we loop the traffic back into the HTTP proxy.

696i1CAC2A4B9CB5CB3F.jpg

Using this configuration, you still have two virtual servers, but you only need to publish one proxy service entry point. Best of all, you can still utilise Stingray’s' HTTP optimization and caching abilities on the loop back virtual server.

698iF8F5A39655DBBD49.png

Comments
by mbodding
on ‎01-28-2016 03:12 AM

Comment from: michael [Zeus Dev Team]
There is one aspect of the CONNECT method that will not work with the rule given in this article. In section 3.3 of his draft, Luotonen describes an optimization he calls 'data-pipelining'. This allows the client to send the first block of data of the tunneled protocol (intended for the remote peer) before it has received a succesful response from the proxy. For example:

 

CONNECT secure.zeus.com:443 HTTP/1.0
User-Agent: Homegrown
first line of data


Note the blank line between the header(s) and the body, the same format that is also used for POST requests. Since ZXTM discards all data received from the client when the TrafficScript function request.sendResponse() is called, this 'pipelined' data would never reach the server. One way to be fully compliant with the specification is to modify the request rule to detect the presence of extra data and to delegate sending the 'Connection established' message to an accompanying response rule. Here's the modified request rule:

 

# The CONNECT request ends with an empty line; the \r is optional:
$request = request.getLine("\r?\n\r?\n");
# Check for SSL connect requests. If it's not a CONNECT then break if( ! string.regexMatch($request,"CONNECT (.+?)Smiley Sad[0-9]+?) HTTP.*") ) break;
# This is an SSL request $host = $1; $port = $2;
# Resolve the hostname to an IP address $ip = net.dns.resolveHost( $host ); if ( $ip == "" ) { log.info("Unknown host: '" . $host . "'"); request.sendResponse("HTTP/1.1 404 Unknown Host\r\n\r\n"); }
# Select the destination node, and tell the client we're connected. pool.select("Forward Proxy Pool", $ip, $port);
# Everything after the first empty line must go to the remote # destination ('back-end'), but everything up to the first line # is only used by the proxy (ZXTM). $to_skip = string.len($request);
# Read in the rest of the request (if any). Do not append, # however, as request.get() gives the full string again $request = request.get();
if( string.len($request) == $to_skip ) { request.sendResponse("HTTP/1.1 200 Connection Established\r\n". "Proxy-Agent: ZXTM/Request\r\n\r\n"); } else { request.skip($to_skip); # tell the response rule that it must send the # 'Connection established' status line connection.data.set("must_send_200", 1); }


And here is the response rule:

 

if( connection.data.get("must_send_200") ) {
   connection.data.set("must_send_200", "");
   $resp = response.get();
   $resp = "HTTP/1.1 200 Connection Established\r\n".
      "Proxy-Agent: ZXTM/Response\r\n\r\n" . $resp;
   response.set($resp);
}

 

The spec is available here: http://tools.ietf.org/html/draft-luotonen-ssl-tunneling-01.txt