Intra-Host VM communication(layer 2) failure due to reversed vNIC IP Configurations on GuestOS

Products

• VMware vSphere ESXi

Issue/Introduction

• Two Virtual Machines (VMs) residing on the exact same ESXi host and within the same Layer 3 subnet are unable to communicate or ping each other.
• Network captures using the pktcap-uw utility show ARP requests successfully leaving the VM. 
• However, trace commands for the affected vNIC (e.g., eth0) indicate that packets are departing from the physical uplink instead of traversing locally within the vSwitch. 
• Packet captures at the switchport connected to eth0 record a source MAC address that belongs to eth1.

Environment

VMware ESXi 
VMware vCenter Server 

Cause

• This is a Guest OS issue.
• The IP addresses configured within the Guest OS were reversed across the dual vNICs ( For e.g. the IP intended for eth0 was assigned to eth1, and vice versa).
• Consequently, traffic is sent out the incorrect vNIC onto a portgroup with a mismatched VLAN. Since the traffic enters a different Layer 2 broadcast domain, the vSwitch forwards the frame to the physical uplink.
• The physical switch then drops the packet because the destination MAC address is not learned on that VLAN.

Resolution

• Identify the current hardware MAC addresses assigned to the VM's network adapters via the vSphere Client.
• Log into the Guest OS of the affected VM and review the IP address configuration for each interface (e.g., using ipconfig /all or ip a).
• Compare the Guest OS MAC-to-IP mapping against the vSphere Client MAC-to-Portgroup mapping.
• Correct the configuration using one of the following methods:
  
  • vSphere UI Reconfiguration: Edit the Virtual Machine settings in the vSphere Client and assign the correct portgroup network assignments for the affected vNICs to match the Guest OS IP layout.
  • Guest OS Reconfiguration: Modify the network settings inside the Guest OS to assign the correct IP addresses to the specific hardware MAC addresses of the intended vNICs.
• Re-test communication between the VMs to confirm the issue is resolved.

 

Additional Information

Troubleshooting virtual machine network connection issues