VM deployed in NSX-T fails to communicate on segment for approx. 10 minutes

Products

• VMware NSX

Issue/Introduction

• After a VM has been deleted and redeployed using the same IP address, connectivity fails on the redeployed VM.
• The redeployed VM may be a newly deployed VM or existing VM.
• The connectivity fails for VM's on different subnet/segments.
• The redeployed VM can ping and respond to pings from IP addresses on the same segment, on the same or different hosts.
• The redeployed VM fails to respond to pings from IP address's in a different segment/subnet.
• The redeployed VM has a new mac address, but uses the same IP address as the deleted VM.
• Using packet capture tools, no traffic is observed leaving the new VM when not in same subnet/segment.
• Listing the mac addresses known to the VDR (Virtual Distributed Router), still shows the deleted VM's mac address:

net-vdr -Nl <vdr-instance-ID>

DR ########-####-####-####-########## ARP Information :
Legend: [S: Static], [V: Valid], [P: Proxy], [I: Interface]
Legend: [N: Nascent], [L: Local], [D: Deleted], [K: linKlif]
Legend: [A: AutoRefresh]

Network Mac Flags Expiry Uptime SrcPort Refcnt Lif UUID
------- --- ----- ------ ------ ------- ------ ---------
<snip>
192.x.x.x ##:##:##:##:##:3d V 441 159 0 6 ########-####-####-####-##########e8

Note: To find the VDR instance name used in the command above, use: net-vdr -Il

Environment

VMware NSX

Cause

The Virtual Machine's  operating system has failed to send out a GARP (Gratuitous ARP) packet when reusing the same IP.
The GARP will refresh all stale IP/MAC entries in NSX and is required by the VDR to update it's own IP to MAC address mappings.

Resolution

The guest operating system is responsible for broadcasting a Gratuitous ARP (GARP) upon the configuration of an IP address. This mitigates conflicts that arise when an IP address has been recently mapped to a disparate MAC address. Regardless of prior IP-to-MAC bindings, transmitting a GARP during IP assignment is an essential networking best practice to ensure upstream ARP caches are accurately updated.


Workaround:

From the impacted VM, that is the redeployed VM, you can manually run an 'arping' command to update all devices of the new IP/MAC mapping:

arping -U -I eth0 192.x.x.x -c 3

The IP address in the command should be the IP address of the redeployed VM and will result in a GARP being sent out with source and destination IP being the same and that of the newly deployed VM.