High latency to the Cloud/Internet destination when routed via VeloCloud SD-WAN

Products

• VMware SD-WAN by VeloCloud

Issue/Introduction

Symptoms:
- Pings from the LAN network of the VeloCloud Edge to a Cloud/Internet destination exhibit high latency.
- Applications and web pages load slowly.

Environment

VMware SD-WAN by VeloCloud

Resolution

This article covers the basic troubleshooting steps for establishing the root cause of high latency.  Specifically, these steps seek to establish whether the cause points to an ISP issue or a VeloCloud issue which would require further investigation with Support.

1. Verify whether there is high latency for the "Direct" path (i.e. Direct break out from the Edge to the Internet).
   1. On the VeloCloud Orchestrator for the site experiencing latency, navigate to the Remote Diagnostics page and scroll down to Traceroute.
   2. In the Destination field, enter any Internet destination (i.e. 8.8.8.8 or 8.8.4.4)
   3. For the Traceroute Using option, select the WAN interface of the Edge
   4. Run the traceroute diagnostic and verify the latency of each hop.
   5. If there is high latency for Direct traffic, the cause of the latency may be the result of an ISP issue and the customer should follow up with the provider.

​​​​

2. Verify whether there is high latency for Multipath traffic that uses a VeloCloud Gateway.

   1. For the same Remote Diagnostics->Traceroute

   2. Enter the same Destination as in Step 1.

   3. For Traceroute Using option, select "VeloCloud Gateway".

   4. Run the traceroute and compare the result, examine the latency of each hop.

   5. If high latency is observed only for the traceroute via the VeloCloud Gateway, proceed to Steps 3 and 4.

 

3. Verify whether there is high latency on the VCMP (VeloCloud Management Protocol) tunnel between the Edge and the Gateways.
   1. Staying on the same Remote Diagnostics page, navigate to the List Path diagnostic.
   2. For Peer, select “Gateway”.
   3. Run the test and examine the latency reported for each Gateway.
   4. Note the IP addresses of each Gateway from the Remote IP section of the List Path output.
   5. If the latency reported for the VCMP tunnel is high, run a Traceroute with the Destination the Primary Gateway IP, and the Traceroute Using option as the WAN interface of the Edge as mentioned in step#4      

 

4. Verify whether there is high latency for the Primary Gateway IP when a traceroute uses the  "Direct" path.

1. Remote Diagnostics page and scroll down to Traceroute.
2. In the Destination field, enter the destination IP as Primary Gateway IP Address
3. For the Traceroute Using option, select the WAN interface of the Edge
4. Run the traceroute diagnostic and verify the latency of each hop.
5. If there is high latency for Direct traffic, the cause of the latency may be the result of an ISP issue and the customer should follow up with the provider.

Note: If you are a partner, the Primary Gateway IP is found on the Monitor->Edges page for that customer by selecting View Gateways for that site.
If you are a customer, please check with your partner to get this information.  If you do not have a Support Partner, check with Support for that information.

 

 

Summary:
If the Direct path test results show high latency (Steps 1 and 4) this points to an ISP issue. 

An additional troubleshooting option is to examine the traceroute and identify the hop with the critical level of latency and which ISP is associated with that hop.  Many of the Tier 1 ISPs offer the Looking Glass tool to debug possible peering issues. You can use an Internet search engine to determine if the ISP has a Looking Glass tool.

Whether you decide to use Looking Glass for further troubleshooting analysis or not, please follow up with the ISP using the traceroute results from Step 1 & Step4.

If the Direct path test results show normal latency, but you are observing high latency via the Gateway , please collect the diagnostic bundle from the Edge and open a case with VeloCloud Support for further troubleshooting.