VMware VeloCloud SD-WAN scenarios where a business policy rule sending traffic "Direct" won't work

Products

• VMware VeloCloud SD-WAN

Issue/Introduction

To explore the scenarios where a business policy rule sending traffic Direct won't work.

Environment

VeloCloud SD-WAN supported versions

Resolution

First things first, verify the problem:

• Check debug.py --flow_dump, either locally on the edge CLI or via the diag bundle. It will show whether or not a specific flow is being sent direct or via the gateway
• Check the flow dump in Remote Diagnostics. It will specifically state which business policy is being hit. One reason the traffic may be going multipath is it's hitting a different BP rule than the one you expected. In those scenarios you need to troubleshoot why the flow is matching what you consider an unexpected BP rule.

 

For scenarios where it's matching the right BP rule and still going Multipath, there are a couple of possibilities: 

• If the Edge has received any secure default routes or specific secure routes from the Partner gateway, then the business policy created to send this traffic "direct" won't work and the traffic will be forwarded only via the gateway path. Check the debug.py --routes and look for any secure routes / default routes present for this flow. Routes with flags PSR or PBSR are secure routes. Note: PG routes can be configured as secure or non-secure. If they are non-secure than the BP can override the PG route. (This is configured as a "Secure BGP Route" or an encrypted static route.)
• Any route received from another edge is also considered a "secure" route and the traffic will be sent to the other edge via VCMP. Our SD-WAN solution does not support the ability to send site-to-site traffic Direct. It always has to be through VCMP. (Unless it gets to the other site via being routed by a static or dynamic route.)
• Another scenario, where the business policy to send traffic direct won't work is if the application-map has an entry "dropIfPartnerGatewayDown": 1," . In this case, the traffic will be always forwarded to the  partner gateway irrespective of what is configured on the business policy. If the path to Gateway is down, we will drop this traffic and will never be sent direct. For some partners we can see all the app-map entries are configured with "dropIfPartnerGatewayDown": 1,"  so for those partners we cannot send the traffic direct.

There's two appmap flags that sound similar but do different jobs. Don't get them confused:

mustUseGateway: Must use gateway if business policy says must use multi-path. If gateway path goes down, it will switch to direct. SD-WAN edge will switch direct traffic back to the VCG path once it re-establishes tunnel to SD-WAN gateway. This flag will not override a BP rule sending traffic direct.

dropIfPartnerGwDown: Must use partner gateway, if path to gateway is down, packets will be dropped. This flag will override a BP rule sending traffic direct.