Scenario when the flow is matching with cloud route but being sent out Direct.

Products

• VMware SD-WAN by VeloCloud

Issue/Introduction

If the tunnel towards the primary VeloCloud gateway was down when the Internet flow was initiated, the traffic will choose the direct path. The traffic will continue using direct path to avoid disruption even after the tunnel to the VCG comes back up.

Environment

VMware SD-WAN by VeloCloud

Resolution

If flow is matched with cloud default cloud route from GW (Gateway), flow can be sent out Direct or Multi-path (via GW) based on Biz policy rule. But if a flow (set to be sent out via Multi-path in biz policy rule), is initiated when there was no cloud default route available (probably because tunnel to GW was down), flow will be sent out direct and VCE will continue to send this traffic via direct.  

 

In the example below, flow (destined for 8.8.4.4) started during primary GW tunnel flap and now being sent out direct even when default cloud route from primary GW is re-installed.  Please note that you need secure CLI access to run the commands from CLI.

 

velocloud 520:~# debug --flow_dump all all all | grep 8.8.4.4 

3           0       0      0               0           0                0   10.125.0.162          8.8.4.4       9777           0       1     normal               APP_ICMP(70)         APP_CLASS_NETWORK_SERVICE(13)   transactional      direct    bw_balance         N/A   00000001-      0x4000a000002L         1   local   0x7ff174005aa0   0x158d10a0   0x162914a0 

 

 

velocloud 520:~# debug --routes 8.8.4.4 

Address    Netmask    Type        Gateway   Next Hop Name                            Next Hop ID   Destination Name                          Dst LogicalId   Reachable   Metric   Preference   Flags   Vlan   Intf   Sub IntfId   MTU   SEG 

0.0.0.0    0.0.0.0   cloud   192.40.64.46    vcg124-usca3   2e4028c0-0000-0000-0000-000000000000       vcg124-usca3   2e4028c0-0000-0000-0000-000000000000        True      255            0       v      0    any          N/A   N/A     0 

0.0.0.0    0.0.0.0   cloud   10.33.33.253             N/A                                    N/A                N/A                                    N/A        True        3            0       S      0    GE1          N/A   N/A     0 

P - PG, B - BGP, D - DCE, L - LAN SR, C - Connected, O - External, W - WAN SR, S - SecureEligible, R - Remote, s - self, H - HA, m - Management, n - nonVelocloud, v – ViaVeloCloud 

 

Now, if you check flow_route_dump for this flow, you will see the route it’s matching to is still default cloud route from GW. This is expected behavior because this flow is still matching with that route, but since the flow entry is created with direct, it’s pushing this traffic via direct. 

 

velocloud 520:~# debug --flow_route_dump all 8.8.4.4 all 3 

Address             Netmask    Type        Gateway                            Next Hop ID                          Dst LogicalId   Reachable   Metric   Preference   Flags   Vlan          Intf   Sub-Intf-Id   MTU 

10.125.0.0    255.255.255.0     any            any                                    N/A                                    N/A        True        0            0      CS      1   br-network1           N/A   N/A 

0.0.0.0             0.0.0.0   cloud   192.40.64.46   2e4028c0-0000-0000-0000-000000000000   2e4028c0-0000-0000-0000-000000000000        True      255            0       v      0           any           N/A   N/A