NAT not working as expected when both SNAT and DNAT rules are configured in NSX for the same flow.

Products

• VMware NSX

Issue/Introduction

• Both SNAT and DNAT rules are configured in NSX to have both the source and the destination IP addresses translated for the same traffic flow.
• The TCP SYN packet is observed to reach the destination with both source and destination IP having been translated.
• The response TCP SYN/ACK packet does not reach the source and instead a TCP RST is observed returning to the destination instead of the expected TCP ACK to complete the handshake.

• This affects TCP traffic, but not stateless traffic like ICMP. 

 

Environment

VMware NSX

Cause

This is expected behavior when trying to apply both a DNAT and an SNAT NSX rule to the same stateful traffic flow in default NSX. 

Resolution

There are two known workarounds for this NAT-based workflow, detailed below;

1. To have both NAT's (SNAT and DNAT) applied to the same outbound packet, which requires double inspection on the response traffic, refer to the workaround detailed in KB NAT not working on NSX Tier0 router with or without IPSec VPN involved.
2. The other option is to configure the NSX DNAT rule to have a source IP or range that includes the post-SNAT IP. However, this option creates a hairpin between the T1 and T0 which creates additional considerations if this configuration is utilized. 

Workaround

• Example SNAT and DNAT rule configuration for option 2 above.
  
  • Note the 'Source IP' of the DNAT is set to a range including the after-SNAT rule IP. 

• With these rules configured, full 3-way TCP handshake completes successfully between source and destination. 
• However, in the traceflow for this we observe only one NAT, the SNAT, being applied initially because during the first inspection the source IP of the TCP SYN doesn't match the configured 'Source IP' of the DNAT rule. 
  • Then the packet is routed to the T0.
    • If the T1 has 'All NAT IP's' enabled for Route Advertisement then the T0 will have a more specific route to the pre-DNAT destination IP and route the packet back to the T1, creating a hairpin. 
    • Then the T1 receives the packet again and parses the NAT rules again. Now the packet matches the configured DNAT since the source IP has been NAT'd by the SNAT during the first phase. 
    • After the DNAT is applied, the packet is forwarded to the T0 again, which then routes this north to the physical network per its route table (BGP or otherwise). 

• This hairpin between the T1 and T0 may cause additional overhead considerations, more so if they are in different Edges and those Edge VMs are running on different ESXi hosts. 

 

Additional Information

See KB NAT not working on NSX Tier0 router with or without IPSec VPN involved which details a workaround that prevents the above T1 and T0 hairpin for scenarios where both SNAT and DNAT are needed.