Error: "The vMotion failed because the destination host did not receive data from the source host on the vMotion network"
search cancel

Error: "The vMotion failed because the destination host did not receive data from the source host on the vMotion network"

book

Article ID: 323392

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

This article provides troubleshooting steps to resolve vMotion connectivity failures encountered during migration attempts.

  • This issue is typically seen when attempting a vMotion migration between two ESXi hosts, and the task fails around **36% (or less)** completion. The failure occurs because a large volume of data is moved, and the destination host does not receive an acknowledgment receipt from the source host on the vMotion network.
  • Network misconfiguration can cause random vMotion failures. In such cases, retrying the vMotion operation may occasionally be successful, but the underlying issue persists.
  • When the issue occurs, the following errors may be seen:

    The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct.

    T    he vMotion migrations failed because the ESX hosts were not able to connect over the vMotion network. Check the vMotion network settings and physical network configuration.

Environment

  • VMware vSphere 7.x
  • VMware vSphere 8.x

Cause

Network misconfigurations are typically the cause for these error messages, such as:

  • Inconsistent vMotion VMkernel adapter settings between the source and destination hosts.

  • Mismatched TCP/IP stacks for vMotion. For example, some hosts might be configured with the "default" TCP/IP stack while others use a dedicated "vMotion" stack.

  • Physical switch port for vmnic the vMotion vmkernel is using is not configured to allow the appropriate vlan tagging for vMotion traffic

  • A common cause for vMotion failures is when the vmkpings fails between source host and destination host and the ESXi host successfully transmits the ICMP request through the physical NIC (vmnic), but the physical network fails to return the reply. This indicates the packet is being dropped or misrouted after leaving the ESXi environment. For more information, see Packet capture on ESXi using the pktcap-uw tool

  • Network packet capture shows that while the ESXi hosts attempted to initiate communication, there was no ARP response received for the target vMotion VMkernel IP addresses.
    37 46.840772   VMware 67:b1:85     Broadcast                     ARP        60 Who has x.x.x.x (Taregt IP)? Tell x.x.x.x (Source IP)
39 47.841863   VMware_67:b1:85     Broadcast                     ARP        60 Who has x.x.x.x? Tell x.x.x.x
41 49.051437   VMware_67:b1:85     Broadcast                     ARP        60 Who has x.x.x.x? Tell x.x.x.x

43 50.263101   VMware_67:b1:85     Broadcast                     ARP        60 Who has x.x.x.x? Tell x.x.x.x
44 51.473162   VMware_67:b1:85     Broadcast                     ARP        60 Who has x.x.x.x? Tell x.x.x.x
45 51.843262   VMware_67:b1:85     Broadcast                     ARP        60 Who has x.x.x.x? Tell x.x.x.x

Resolution

Review the network configuration stack.

  1. Check for IP address conflicts on the vMotion network and if the vmk configuration is correct.

    Note:     Each host in the cluster should have a vMotion vmkernel with an assigned unique IP address.

  2. Check for IP connectivity and packet loss over the vMotion network. Try having the source host ping (vmkping) the destination host's vMotion vmknic IP address for the duration of the vMotion.

    Example

    vmkping -I vmk# <IP of destination host vMotion vmk>

    where vmk# is the vmkernel number of the vMotion vmkernel on the source host. For more information, see Testing VMkernel network connectivity with the vmkping command.

  3. Check the driver/firmware of the physical adapter used for vMotion.

    Note: If the driver or firmware is not up to date, the vMotions can fail with a "timeout" or "waiting for data" error. For more information, see Determining Network/Storage firmware and driver version in ESXi.

  4. Ensure the ESXi firewall ruleset for vMotion is enabled and that there are no other conflicting rules blocking the traffic.

  5. Check for potential interaction with firewall hardware or software that prevents connectivity between the source and the destination TCP port 8000.

  6. Verify all hosts use the same TCP/IP stack for their vMotion VMkernel adapters (either all vMotion stack or all Default stack).

    Check via vCenter: Host > Configure > Networking > TCP/IP stacks.

  7. Engage physical network team if the packets are dropped in physical network to isolate the drop point of packets in the network. 

  8. Lack of ARP resolution indicates a Layer 2 connectivity issue within the physical network infrastructure or virtual switch configuration. When an ESXi host successfully transmits an ARP request but does not receive a reply, the physical network is failing to return the expected traffic, causing the migration to timeout and report the host as "down". It is recommended to engage Physical network team.




Additional Information

Other Examples

  • Migration [184329483:<mig_id>] failed to connect to remote host <###.###.###.###>: Timeout.

    This error indicates that remote host did not accept the connection within the allowed time limit.
     
  • Migration [-1408237366:<mig_id>] failed to connect to remote host <###.###.###.###>: Connection refused.

    This error indicates that the remote host is explicitly not listening on the vMotion port. The vMotion port is 8000, see also Port requirements for VMware vSphere ESXi

Additional Articles

Powered by Wolken Software