NSX edge low throughput observed when flow-cache enabled with NAT configured on Tier-0 or Tier-1 gateway

Products

• VMware NSX

Issue/Introduction

• Throughput and / or connectivity impact observed for traffic traversing impacted edge nodes
• Download speeds can be reduced to zero bytes/sec.
• Packet capture from edge node host on the edge uplink interface switchport shows invalid checksum. 
  

  pktcap-uw --switchport 33554442 --ip 10.4.4.1 --dir 2 -o -| tcpdump-uw -vvvenr -
  
  <MAC Address> > <MAC Address>, ethertype IPv4 (0x0800), length 1514: (tos 0x0, ttl 57, id 26267, offset 0, flags [DF], proto TCP (6), length 1500)
      <IP Address> > <IP Address>.50002: Flags [.], cksum 0x0000 (incorrect -> 0xd50a), seq 0:1460, ack 1, win 62, length 1460: HTTP
  <MAC Address> > <MAC Address>, ethertype IPv4 (0x0800), length 1514: (tos 0x0, ttl 57, id 26268, offset 0, flags [DF], proto TCP (6), length 1500)
      <IP Address> > <IP Address>.50002: Flags [.], cksum 0x0000 (incorrect -> 0xd50a), seq 0:1460, ack 1, win 62, length 1460: HTTP

Environment

VMware NSX-T Data Center

Cause

A TCP packet whose checksum is zero doesn't get the updated checksum after NAT when the packet hits a valid flow cache entry.  Any checksum other than zero works as expected.

Resolution

This issue is resolved in VMware NSX-T Data Center 3.1.2, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

Workaround
For workaround use either of the following options:

Option 1:  Disable NAT
-NSX UI > Networking > NAT

 
Option 2: Disable Flow-Cache
-Run the following commands from edge CLI as admin:

set dataplane flow-cache disabled
restart service dataplane

NB:  Restarting the dataplane service will temporarily impact the existing session flowing through the edge.

 

Additional Information

Impact/Risks:

• Network disconnect or less throughput