Troubleshooting virtual machine connectivity to a Virtual IP (VIP) or static IP address

Products

• VMware vCenter Server
• VMware NSX Data Center for vSphere
• VMware vSphere ESXi
• VMware vSphere Foundation
• VMware vSphere Standard
• vSphere with Tanzu
• VMware vSphere with Tanzu

Issue/Introduction

A VM is connecting to a destination over the same network (layer 2) via a static IP address or VIP.  The VM experiences inconsistent connectivity to either or both.
This is a method for troubleshooting this kind of issue.

Cause

ARP is a layer 2 function.  It is the first step in network connectivity.

Typical resolutions are:

• Power cycle hardware devices
• Update component drivers and firmware
• Reboot virtual machines or hardware components

Resolution

For any network connection, Layer 2 ARP must be successful.  Without this Layer 3 IP will not work.
Validate the ARP is successfully completing.   ARP will issue a "who has" request. 

1. Open a console session in the source VM guest desktop.
2. Start a continuous ping to the IP and or VIP (Destination IP)
3. Open an SSH session to the ESXi host that has the source VM located on it
4. Get the switchport number for the VM's NIC carrying the traffic.  
        esxcli vm process list |less find the VM in the list and record the World ID (WID)
        esxcli network vm port list -w <WID>   Record the switchport number
5. Capture packets using pktcap-uw --switchport <SwitchPort ID> --capture VnicTX,VnicRx  --ng -c 100 -o - |tcpdump-uw -enr - -nn   
   This is a live capture to screen for 100 packets.
6. Observe ARP for requests and replies.  If there are no replies to the "who has" ARP request, then the issue is with ARP. This is an issue within the infrastructure.
7. If replies are present, then this is not an ARP issue and further troubleshooting is needed at the higher OSI layers.