Intermittent Connectivity for External Traffic to NSX Segments via Palo Alto VM Across Different Hosts

Products

• VMware NSX

Issue/Introduction

• External users experience timeouts/latency when accessing VMs on NSX segments via a Palo Alto VM.
• The issue does not occur if the test client is a VM already inside an NSX segment.
• The issue does not occur if the Palo Alto VM and destination VM are on the same host.
• Symptoms include TCP checksum errors and retransmissions in packet captures.

Environment

VMware NSX 4.x
ESXi 8.0U3
Cisco UCS Hardware
Palo Alto Firewall VM

Cause

The physical NIC or network fabric is likely mishandling TCP checksums for traffic that originates outside the NSX domain and is then encapsulated into Geneve to reach a different host.

Resolution

Run the following command to check if the issue is caused by the NIC HW Geneve offload by switching to software simulation of Geneve offload:

esxcli network nic software set --vmnic=vmnicX --geneveoffload 1

If the issue goes away with the software simulation, it indicates that there is an issue with the NIC drivers. However, if the issue persists then, it is external to the ESXi hosts/NIC drivers and points towards the physical NIC or network fabric.