How to investigate vMotion failures when you see "Failed waiting for data." and/or "Connection closed by remote host, possibly due to timeout. "
Article ID: 379192
Updated On:
Products
Issue/Introduction
vMotion is another term for a "hot migration" of a virtual machine from one ESXi host to another.
To migrate a VM to / from an ESXi host, that ESXi host must have a vmkernel interface (vmk0, vmk1, etc.) that is enabled for the vMotion service.
Environment
VMware vCenter Server 8.x
VMware vCenter Server 7.x
VMware vSphere ESXi 8.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 6.x
Cause
When you see an error containing "Failed waiting for data." and/or "Connection closed by remote host, possibly due to timeout. ", this means that there is no connectivity between the source and destination ESXi hosts along the data path associated with:
a) The vmnic(s) on the source ESXi host being used to carry the vmkernel traffic for the vmk associated with the vMotion service; and
b) The vmnic(s) on the destination ESXi host being used to carry the vmkernel traffic for the vmk associated with the vMotion service
This is typically not a software error.
Resolution
To investigate this:
1) Using the vSphere client, identify which vmkernel interface is "tagged" for vMotion -- this can be done by using the Edit function to reveal the settings for each vmkernel interface in the "Configure" section for the host. Do this for each of the source and destination host. At this time, also determine if there is a VLAN ID specified for the traffic. If so, make a note of it.
2) SSH into the source ESXi host with root privileges
3) Enter esxtop and then lower case "n" for networking
4) Observe which physical uplink (PNic, or vmnic) is associated with the vmk identified in step 1)
5) Using the vSphere client, go to the Configure section for the host and then Physical adapters under Network and see if there is any CDP or LLDP info that would reveal which physical switch and switchport the vmnic identified in step 4) is connected to. If there is no CDP or LLDP information available, that is because the team that manages the physical switch has set up the switch to not provide that info.
6) Repeat steps 2) through 5) for the destination host.
7) Provide the following information to your team that manages the physical switches to which the uplinks identified in step 4) are connected:
a) Source Host name
b) IP address / MAC address of the vMotion tagged vmkernel interface on the Source Host
c) vmnic being used to carry the traffic
d) VLAN ID if specified
e) If available, CDP or LLDP info specifying the physical switch / switchport to which the vmnic is connected
8) Also provide the team that manages the physical switches the following KB article:
VLAN configuration on virtual switches, physical switches, and virtual machines
9) Ask the team to verify that they have configured the switchports correctly based on whether or not you have configured a VLAN ID for the vmk interface.
--> If you have configured a VLAN ID, the switchport must be configured as "Trunk", per the KB.
--> If you have not configured a VLAN ID (i,e, 0), the switchport must be configured as "Access", per the KB.
10) If you have done all of these steps, confirmed that the configuration of the physical switch is consistent with the recommendations of the above KB, please open a Support Case with Broadcom for the Core Networking team. )Reference: Creating and managing Broadcom support cases
Additional Information
Other useful KB references:
Feedback
