I have created a white list of URLs. It is working for explicit request while transparent does not work - why?


Article ID: 166574


Updated On:


Advanced Secure Gateway Software - ASG ProxySG Software - SGOS



  • Explicit and Transparent mix deployment
  • Created white list are working for users when they point their browser via the proxy.  It is failing when they on transparent deployment (e.g. bridging/inline, WCCP)
  • Request are SSL.  Verified that port 443 is set to intercept, transparent under Proxy Services



You are seeing a normal behavior if the request is https over transparent with no SSL interception happening and a Server Name Indication extension is not included in the Client Hello packet.

For Explicit https request, browsers or the application does HTTP "CONNECT" method where the domain name is visible on the request which can be seen by the proxy.

Here is an example of explicit HTTPS request:

No.     Time        Source                Destination           Protocol Info

    396 11.565418s           TCP      61598 > http-alt [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=8 SACK_PERM=1

    397 11.565873s          TCP      http-alt > 61598 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 SACK_PERM=1 WS=6

    398 11.565933s           TCP      61598 > http-alt [ACK] Seq=1 Ack=1 Win=66560 Len=0

    399 11.566089s           HTTP     CONNECT bto.bluecoat.com:443 HTTP/1.1    <<<<<<<<<<   Look

    400 11.566312s          TCP      [TCP Window Update] http-alt > 61598 [ACK] Seq=1 Ack=1 Win=4097472 Len=0

    401 11.623771s          TCP      http-alt > 61598 [ACK] Seq=1 Ack=206 Win=4097472 Len=0

    402 11.692975s          HTTP     HTTP/1.1 200 Connection established

    403 11.693573s           TLSv1    Client Hello

    405 11.773777s          TCP      http-alt > 61598 [ACK] Seq=40 Ack=389 Win=4097472 Len=0

    406 11.782341s          TLSv1    Server Hello

Frame 399 shows the HTTP "CONNECT" method after the TCP handshake.

When the browsers or the applications are requesting content towards an HTTPS site transparently via the proxy, URLs are encrypted on HTTPS payload. Without SSL interception or the presence of the SNI extension, URLs can not be seen and would not match https URL created on your white list.
Here is an example of Transparent HTTP request:

No.     Time        Source                Destination           Protocol Info

   1279 45.863481s        TCP      62057 > https [SYN] Seq=0 Win=8192 Len=0 MSS=1260 WS=8 SACK_PERM=1

   1280 45.949138s         TCP      https > 62057 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 SACK_PERM=1 WS=0

   1281 45.949219s        TCP      62057 > https [ACK] Seq=1 Ack=1 Win=66560 Len=0

   1282 45.949421s        TLSv1    Client Hello  <<<<<<<<<Look

   1283 46.036899s         TLSv1    Server Hello

   1284 46.036977s        TCP     [TCP segment of a reassembled PDU]

Frame 1282 shows SSL/TLSv1 "CLient Hello" after the TCP Handshake.  It is a normal behavior for browser not to send clear text on the wire for HTTPS site when it (browser) is not proxy aware.

NOTE:  Without SSL interception, Explicit can not match HTTPS URLs if the URLs includes URL path.

Good to read SSL interception articles:

TECH241134 - Configure the SSL proxy on the ProxySG for transparent interception and authentication using an SSL certificate issued from a Microsoft PKI server

TECH241741 - How do I filter non-SSL Intercepted HTTPS traffic on a Transparently deployed Proxy

TECH241137 - Configuring SSL interception for Explicit proxy