NSX Edge unable to pass traffic intermittently

Products

• VMware NSX

Issue/Introduction

• The NSX Edge becomes unable to pass traffic after a period of time, not at regular intervals.
• The issue persists until the Edge VM is rebooted, but the issue returns after another period of time.
• The NSX Manager may report no issues with the Edge itself. 
• You may see the following messages on the NSX Edge in the /var/log/syslog directory indicating the nestdb identified connectivity issues or that the dataplane service has stopped:
  grep -ai "duplicate ip" var/log/syslog* | less
[ERROR] Duplicate IP detected (########-####-####-####-############, ###.###.###.###) from ##:##:##:##:##:01  errorCode="EDG0400013"

####-##-##T##:##:##.###Z edge-vm-01 NSX ##### - [nsx@#### comp="nsx-edge" subcomp="node-mgmt" username="root" level="WARNING" invalid="true"] REPEATS: # repeats in ## sec: Error reading datapath config, rc:-1, out:, err:The dataplane service is not running, has failed, or Maintenance Mode is Enabled

Environment

VMware NSX

Cause

Duplicate IP address conflicting with the Edge VM is causing connectivity issues.

Resolution

• Run the below command as root in the Edge to determine if a duplicate IP has been detected on the Edge (press "q" to exit the view):

grep -ai "duplicate ip" var/log/syslog* | less

• If a duplicate has been detected the output should display errors like the below:

[ERROR] Duplicate IP detected (########-####-####-####-############, ###.###.###.###) from ##:##:##:##:##:01  errorCode="EDG0400013"

• Please note the IP and MAC in the above output correspond to the Edge, not the component infringing on it. If the duplicate IP has been added to the environment fairly recently, the below command may provide the infringing component's MAC:

grep -B 10 -ai "Duplicate IP detected" var/log/syslog* | grep -E "neigh entry|Duplicate" | grep -A 1 "neigh entry"

• Shortly before a "Duplicate IP detected" error, there may be a neighbor entry log as the new component with the same IP is added, but with a different MAC than what is reported in the IP error:

SWITCHING [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="neigh" tname="dp-ipc31" level="INFO"] announce type neigh entry (########-####-####-####-############, ###.###.###.###) with ##:##:##:##:00:02 is created, dad_state T, prefix_len 0

• Identify what component the above MAC belongs to and change the IP.

If the above output does not provide the MAC of the infringing component, it may be required to power off the Edge and run a ping or traceroute to the duplicate IP to isolate what it belongs to in order to address the duplicate.