ESXi host shows incorrect or stale ARP entry due to duplicate IP address

Products

• VMware vSphere ESXi

Issue/Introduction

• An ESXi host shows an incorrect ARP entry that will not clear on its own.
• The underlay network and other ESXi devices possess the correct MAC address.
• The ARP table on the affected ESXi host displays a VMware OUI MAC address that you do not expect to see assigned to the IP:
  • esxcli network ip neighbor list
  • You see output similar to:

    Neighbor         Mac Address        Vmknic        Expiry  State  Type
    ---------------  -----------------  ------  ------------  -----  ----
    192.168.#.#      00:50:56:##:##:##  vmk2        #### sec         Dynamic

Cause

A duplicate IP address exists in the environment.

Another node (such as a virtual machine or VMkernel adapter) is configured with the same IP address, causing the ESXi host to learn and cache the "incorrect" MAC address.

Resolution

Identify the virtual machine using the conflicting MAC address by querying the vCenter Server database (VCDB).

1. Open an SSH session to the vCenter Server appliance and execute the below command.  Be sure to replace 00:50:56:##:##:## with the MAC you're looking for:

   1. /opt/vmware/vpostgres/current/bin/psql -h localhost -U postgres -d VCDB -c "select e.name as vm_name, n.mac_address as mac_address, e.id as vm_id, v.file_name as vm_file, h.dns_name as esxi_name, h.ip_address as esxi_ip from vpx_entity e, vpx_vm v, vpx_host h, vpx_nic n where e.id=(select entity_id from vpx_nic where mac_address='00:50:56:##:##:##') and v.id=e.id and v.id=n.entity_id and v.host_id=h.id;"

2. Correct the IP address configuration on the identified virtual machine or VMkernel adapter to resolve the duplicate IP conflict.

3. (Optional) Clear the stale ARP entry on the affected ESXi host by running the below command via SSH.  Be sure to replace <IP_to_remove> with the IP you wish to remove (do not include the angle brackets < > in your command):

   1. esxcli network ip neighbor remove -v 4 -a <IP_to_remove>
      

   2. Note: Clearing the ARP entry without resolving the underlying duplicate IP assignment will result in the incorrect ARP entry returning the next time the conflicting MAC transmits traffic to the host.

Additional Information

• Find VM by MAC address querying the VCDB
• Clearing the ARP cache on a VMware ESXi host