Dataplane impact using Active/Active Stateful services and 2 layers of T0s

Products

• VMware NSX

Issue/Introduction

• The environment has 2 layers of Tier-0 (T0) routers:
  T0 (A/A) Stateful (Edge cluster 1) ---- T0 (A/S) Stateful (Edge cluster 2) ---- T1 (A/S) Stateful (Edge cluster 2) 
• Traffic is dropped by the Gateway Firewall on the A/A Stateful T0

Environment

VMware NSX

Cause

To achieve Active Active stateful services, Edges may need to redirect(punt) traffic to the Edge which is tracking the stateful traffic flow.
The default redirection policy is IP_HASH.
This policy supports NAT on the stateful A/A T0 however it does not support traffic between the uplinks of T0.
Since 2 layers of T0s requires traffic between T0 uplinks, the default policy cannot support this designed.

To allow for a second Tier-0 layer, a redirect policy of SRC_DST_IP_HASH must be used.
Note this policy does not support NAT on the A/A stateful T0 router.

In summary:

• Redirection policy of IP_HASH supports NAT but does not support a second T0 layer.
• Redirection policy of SRC_DST_IP_HASH supports a second T0 layer but does not support NAT.

Resolution

Using a REST API Client or curl:

1. Query the current configuration of the Stateful A/A T0 
   GET https://<manager>/policy/api/v1/infra/tier-0s/<T0 router name>
2. Copy the full body output from 1) and edit the redirection policy 
       "stateful_services": {
        "enabled": true,
        "redirection_policy": "IP_HASH"    ---->   "SRC_DST_IP_HASH"
    },
3. Using the full modified API body update the T0 with the POST API including the header content-type application/json
   POST https://<manager>/policy/api/v1/infra/tier-0s/<T0 router name>

Additional Information

Intermittent packet drops with Active/Active T0 and Stateful Firewall Rules