TCP communication from on-premises machines to migrated VMs times out after migrating VMs connected to an L2 extended segment to the cloud

Products

• VMware HCX
• VMware NSX

Issue/Introduction

• After transferring approximately 64 KB of data via TCP, subsequent packets are dropped and transfer fails.
• A VM connected to the source L2 extended segment was migrated via HCX to connect to the target L2 extended segment.
• The routing path between the on-premises machine and the migrated VM is asymmetric, passing through the L2 extension on either the forward or return path only.
• Gateway Firewall is enabled on the Tier-1 Gateway connected to the target L2 extended segment, and a Stateful Firewall allow rule is configured.

Environment

VMware HCX
VMware NSX

Cause

When using a Stateful Firewall within the NSX Gateway Firewall, if the firewall cannot track TCP ACKs returned for transmitted data, it determines that data transmission exceeding the default TCP window size is occurring and blocks the communication. This behavior impacts asymmetric TCP communication passing through the affected Gateway Firewall.

Resolution

• If asymmetric routing is expected, change the relevant Stateful Gateway Firewall rule to Stateless.
• If asymmetric routing is unexpected, review the routing configurations on the network devices along the path. From an HCX perspective, reviewing KB#395509 is recommended.

Additional Information

KB#395509 : Traffic Not Following HCX Mobility Optimized Networking (MON) Policy Routes As Expected