Users in Egypt unable to connect to Cloud SWG data centers using WSS Agent

Products

• Cloud Secure Web Gateway - Cloud SWG

Issue/Introduction

Global employees accessing internet using Cloud SWG via WSS Agents.

After deploying WSS Agent to users in Egypt, no one could connect to Cloud SWG services.

WSS Agent tunnel failed to establish to all nearest data centers, but also remote data centers using dpOverride that corporate users in other locations could access without issues.

Tethering the same WSS Agent host to some other ISPs in the region allowed the tunnel to establish.

 

Environment

WSS Agent.

ISPs intercepting traffic.

Cause

ISPs intercepting traffic and blocking protocols it cannot understand.

Resolution

Work with local ISPs to get permission to tunnel traffic into Cloud SWG destination IP addresses.

In scenario below, ISP was doing protocol detection on initial payload exchanged from client side, not recognising the protocol and not forwarding requests upstream. As a result, the Cloud SWG tunnel endpoint never returned the full authentication challenge needed to establish the tunnel. 

Additional Information

Symdiag PCAPs give key information regarding the flow. What we see from the PCAPs is the following:

- User generates a CTC request and gets list of nearest data centers
- User tried to access nearest data center using UDP port 443 and fails to complete the handshake
- User switches to TCP transport and then tries to establish a tunnel to each of the returned data center VIPs
- after failing to connect to all 3 returned data centers, the WSS Agent errors out.

Of more importance is the initial application traffic via the WSS Agent. From screenshot below:

• The WSS Agent sends 2 requests over UDP tunnel of length 64 and 203 bytes without any response back. When all works fine, the Agent should get full payloads back from the Cloud SWG VIP with authentication specific information.
• After 6 seconds of inactivity, the WSS Agent switches to TCP 443 to see if it can bring up tunnel
• It sends an initial request over TCP tunnel (similar payloads as UDP tunnel) but never gets any of expected response back ..
  • WSS Agent sends a TCP FIN after 10 seconds of inactivity
• Process continues over and over again until WSS Agent errors out