Troubleshooting vMotion network connectivity issues

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides information about the tools that can be used to troubleshoot vMotion network connectivity issues.

Environment

VMware vSphere ESXi 6.7.x
VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 8.0.x

Resolution

To troubleshoot network connectivity issues for vMotion:

1. Identify which vmknics are used for vMotion on both hosts.

   
   Option 1 Using the vCenter Server User Interface (UI)

   
   
   
   
   Notes: All VMkernel adapters with vMotion Enabled will be used for vMotion. For more information on how to enable vMotion on VMkernel adapters, see Creating a VMkernel port and enabling vMotion on an ESXi/ESX host. For best practices on which VMkernel adapters to enable for vMotion, see the Networking Best Practices for vSphere vMotion section of the vCenter Server and Host Management Guide.

   
   Option 2 Using the ESXi command line

   esxcli network ip interface list
    
   Sample output:
   
   vmk0
   Name: vmk0
   MAC Address: 02:##:##:##:##:65
   Enabled: true
   Portset: vSwitch0
   Portgroup: Management Network
   Netstack Instance: defaultTcpipStack
   VDS Name: N/A
   VDS UUID: N/A
   VDS Port: N/A
   VDS Connection: -1
   Opaque Network ID: N/A
   Opaque Network Type: N/A
   External ID: N/A
   MTU: 1500
   TSO MSS: 65535
   RXDispQueue Size: 1
   Port ID: 33554436
   
   For each VMkernel adapter entry, run this command:
   
   esxcli network ip interface tag -i [VMkernel adapter name]
   
   Example command:
   
   esxcli network ip interface tag -i vmk0
   
   Any adapters with tag value vMotion will be used for vMotion.
   
   Sample vMotion VMKernel adapter value:
   
   Tags: VMotion
    
2. Run diagnostic tools between source and destination vMotion VMkernel adapters
   
   vmkping
   
   vmkping uses a VMkernel’s TCP/IP stack to send ICMP traffic to a destination host.
   
   Suppose you have source vMotion VMkernel adapter vmk0 with IP XX.XX.XX.1, and destination vMotion VMkernel adapter vmk0 with IP XX.XX.XX.2
   
   From source ESXi host, run this command:
   
   vmkping -I vmk0 xx.xx.xx.2
   
   Similarly, from destination ESXi host, run this command:
   
   vmkping -I vmk0 xx.xx.xx.1

        If the vmotion is configured on vmotion tcp/ip stack, run the below command:

        vmkping -I vmk0 xx.xx.xx.1 -S vmotion


Note: After you create a VMkernel adapter on the vMotion TCP/IP stack, you can use only this stack for vMotion on this host. The VMkernel adapters on the default TCP/IP stack are deactivated for the vMotion service.

 

For more information on vmkping, see Testing VMkernel network connectivity with the vmkping command.

OR

Run this command: vmkping --help

With long distance vMotion, the maximum supported RTT is 150 miliseconds.

nc

nc is the netcat utility and can be used to verify connectivity on a specific remote port.

Suppose you have source vMotion VMkernel adapter vmk0 with IP xx.xx.xx.1, and destination vMotion VMkernel adapter vmk0 with IP xx.xx.xx.2. Since vMotion listens on port 8000 of the vMotion VMkernel adapter, you need to verify if you can establish a connection to port 8000.

From source ESXi host, run this command:

nc -z xx.xx.xx.2 8000

From destination ESXi host, run this command:

nc -z xx.xx.xx.1 8000

Sample successful output:

Connection to xx.xx.xx.2 8000 port [tcp/*] succeeded!

For more information on nc and troubleshooting specific tcp/ip port connectivity issues, see:

• Troubleshooting network and TCP/UDP port connectivity issues on ESX/ESXi
• Troubleshooting vMotion fails with network errors

Workaround:
VMware Skyline Health Diagnostics for vSphere - FAQ