Error: "Compatibility check took too long to complete" When Selecting Target Host During Cross vCenter vMotion

Products

• VMware vCenter Server

Issue/Introduction

When attempting to perform a cross vCenter vMotion and selecting a target host, the operation fails with the error message: "Compatibility check took too long to complete. Please check vCenter Server logs for more details." The migration process stops at the validation phase, preventing successful VM migration between different vCenter environments.

Upon checking the logs, entries similar to the following may be observed in the destination host's hostd.log:

[NFC ERROR]NfcNewAuthdConnectionEx: Failed to connect to server <IP_address>:902
[NFC ERROR]NfcEstablishAuthCnxToServer: Failed to create authentication connection
[NFC ERROR]Nfc_BindAndEstablishAuthdCnx3: Failed to connect to server <IP_address>:902

These errors indicate connection failures on port 902, which is critical for vMotion operations.

Environment

• VMware vCenter Server 7.0 or later
• ESXi hosts 7.0 or later
• Cross vCenter environment with different clusters

Cause

The primary cause of this issue is network connectivity problems between the ESXi hosts involved in the cross vCenter vMotion. Specifically:

1. Port 902 is blocked or unreachable between the source and destination ESXi hosts
2. This port is required for the Network File Copy (NFC) service, which handles VM data transfer during vMotion
3. Firewall rules, NAT configurations, or routing issues may be preventing the direct host-to-host communication required for vMotion

Additional factors that can contribute to this issue include:

• Time synchronization problems between vCenter Servers and ESXi hosts
• Firmware and driver incompatibilities
• Storage connectivity issues between environments

Resolution

To resolve the "Compatibility check took too long to complete" error during cross vCenter vMotion:

1. Verify direct network connectivity between source and destination ESXi hosts:

   1. Test connectivity to port 902 using netcat from the source ESXi host:
      
      nc -z -v <destination_host_IP> 902

   2. If the connection fails, work with your network team to ensure port 902 is open between hosts

2. Ensure all cross vCenter vMotion prerequisites are met:

   1. Review and implement the requirements in KB article 344959 (Cross vCenter Migration and Clone requirements)

   2. Pay special attention to the networking requirements section

3. Check and correct time synchronization:

   1. Configure all vCenter servers and ESXi hosts to use the same NTP servers

   2. Verify NTP synchronization is working

4. Check for NAT configurations:

   1. Note that using NAT between vCenter Server and ESXi hosts is an unsupported configuration with no workaround per KB 344283, "Using NAT between the vCenter Server system and ESXi hosts will disconnect the host from the vCenter"

   2. Ensure direct routing between all components involved in the migration

5. Verify storage connectivity:

   1. Ensure the destination host has proper access to the target datastore
      
      
   2. For NFS datastores, check that the NFS server is accessible from the destination host
      
      
6. Review and address any firmware or driver update requirements:
   1. Update host firmware and drivers to the latest supported versions
      
      
   2. Ensure compatibility between source and destination hosts

Additional Information

1.  

• For additional troubleshooting of connectivity issues, you can capture and analyze network traffic during a migration attempt using the pktcap-uw utility. Replace the vmk"#" with the actual number of the vmk used for management for the affected ESXi host:

  pktcap-uw --vmk vmk0 --dir 2 --outfile /tmp/vmk0_capture.pcap
  

• MTU mismatches can also cause connectivity issues. Ensure consistent MTU settings across your virtual and physical network infrastructure.