VM deployed in NSX-T fails to communicate on segment for approx. 10 minutes

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

• After a VM has been deleted and redeployed using the same IP address, connectivity fails on the redeployed VM.
• The redeployed VM may be a newly deployed VM or existing VM.
• The connectivity fails for VM's on different subnet/segments.
• The redeployed VM can ping and respond to pings from IP addresses on the same segment, on the same or different hosts.
• The redeployed VM fails to respond to pings from IP address's in a different segment/subnet.
• The redeployed VM has a new mac address, but uses the same IP address as the deleted VM.
• Using packet capture tools, no traffic is observed leaving the new VM when not in same subnet/segment.
• Listing the mac addresses known to the VDR (Virtual Distributed Router), still shows the deleted VM's mac address:

net-vdr -Nl <vdr-instance-ID>

DR 1e85f6db-f088-4f07-89e7-b7865220ede2 ARP Information :
Legend: [S: Static], [V: Valid], [P: Proxy], [I: Interface]
Legend: [N: Nascent], [L: Local], [D: Deleted], [K: linKlif]
Legend: [A: AutoRefresh]

Network Mac Flags Expiry Uptime SrcPort Refcnt Lif UUID
------- --- ----- ------ ------ ------- ------ ---------
<snip>
172.16.20.10 00:50:56:9d:c1:3d V 441 159 0 6 e5063480-f5ef-4d02-837e-1201beb185e8

Note: To find the VDR instance name used in the command above, use: net-vdr -Il

Environment

VMware NSX-T Data Center

Cause

The VM's operating system has failed to send out a GARP (Gratuitous ARP) packet when reusing the same IP.
The GARP will refresh all stale IP/MAC entries in NSX and is required by the VDR to update it IP address to MAC address mappings.

Resolution

It is the VM operating systems responsibility to send out a GARP when the IP address is added.
This issue will not occur if the same IP address has not been used recently with a different MAC address.
It is always a good practice to send out GARP when IP address is added.

Workaround:
From the impacted VM, that is the redeployed VM, you can manually run an 'arping' command to update all devices of the new IP/MAC mapping:

arping -U -I eth0 172.16.20.10 -c 3

The IP address in the command should be the IP address of the redeployed VM and will result in a GARP being sent out with source and destination IP being the same and that of the newly deployed VM.