T1 Gateway Firewall dropping SYN packets with source port reuse in quick succession

Products

• VMware vDefend Firewall

Issue/Introduction

You are using NSX-T.

A firewall rule is in place to allow traffic on the T1 gateway between source and destination.

The destination IP is used in the destination field on the allow rule (Note: not reproduceable with an ANY/ANY rule).

The source is reusing the same port in quick succession.

There is a DNAT rule present on the T1 (Note: DNAT does not need to be related to the above flow).

 

Cause

This is caused when the new flow uses the same source port in very short intervals (approximately 1 second). There is a timing window where the logic incorrectly forwards the SYN packet only on the egress interface and misses the ingress processing and so new state creation doesn't occur.   As a result, the returning SYNACK is not correctly passed back to the client as the traffic is dropped.

To confirm use the below curl. Run command twice, within a 1 seconds gap between each command. If the issue is present, the connection will time out:

curl -IL -H "Connection: close" --local-port <source_port> <destination_IP> --connection-timeout 5

Resolution

This is a known issue impacting VMware NSX.

To workaround this issue, use stateless rules on the gateway firewall, remove the DNAT rule or remove the IP from the source/destination of the firewall rule.