High network latency between NSX overlay and physical network due to Promiscuous Mode

Products

• VMware NSX

Issue/Introduction

• You may experience high network latency between systems on the NSX overlay and the physical network, or between systems on two Tier-1 (T1) gateways located on the same site.

• This latency is particularly noticeable when traffic increases during standard operations or heavy workloads, such as backups.

• Additionally, you may observe that Edge virtual network interfaces (vNICs) receive two or more times the amount of traffic compared to your physical vmnics as observed from net-stats, causing the CPU assigned to be consistently overutilized.
  
  

• You have NSX edge nodes placed in distributed port groups with promiscuous mode enabled.

Environment

VMware NSX

Cause

• This issue occurs because the NSX Edges reside on Distributed Virtual Port (DVPort) groups that have Promiscuous Mode enabled.
• Enabling Promiscuous Mode duplicates traffic, which introduces additional load and leads to performance degradation.

Resolution

To resolve this issue, perform the following steps:

1. Evaluate your environment to determine if Promiscuous Mode is necessary on all of your Edge port groups.

2. If it is not necessary, disable Promiscuous Mode on the Edge port groups.

3. If Promiscuous Mode cannot be disabled, redesign your environment to move those Edges to a different port group, a different cluster, or a different VDS.

If the issue persists even after making above changes to the environment, feel free to open a case with Broadcom Support Team for further investigation.