Cold migrations within the same vCenter are failing with the error: "Cannot connect to host."

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

 

• Migrating or importing a powered-off VM into a new vCenter fails at 39%.
• Migrating a powered-off VM between two hosts within the same vCenter fails.
• Messages in /var/run/log/hostd.log of the source ESXi host indicate that the NFC connection to the destination host failed:

info hostd[PID] [Originator@6876 sub=Libs opID=********-*****-auto-****-h5:********-**-**-****** user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] Cnx_Connect: Error message: Failed to connect to server X.X.X.X:902
warning hostd[PID] [Originator@6876 sub=Libs opID=********-*****-auto-****-h5:********-**-**-****** user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect: Failed to connect to server X.X.X.X:902
warning hostd[PID] [Originator@6876 sub=Libs opID=********-*****-auto-****-h5:********-**-**-****** user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect to peer. Error: Failed to connect to server X.X.X.X:902
warning hostd[PID] [Originator@6876 sub=Libs opID=********-*****-auto-****-h5:********-**-**-****** user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] NfcEstablishAuthCnxToServer: Failed to create new AuthD connection: Failed to connect to server X.X.X.X:902
warning hostd[PID] [Originator@6876 sub=Libs opID=********-*****-auto-****-h5:********-**-**-****** user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] Nfc_BindAndEstablishAuthdCnx3: Failed to create a connection with server X.X.X.X:902 Failed to connect to server X.X.X.X:902
error hostd[PID] [Originator@6876 sub=NfcManager opID=********-*****-auto-****-h5:********-**-**-****** user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] Unable to connect to NFC server: Failed to connect to server X.X.X.X:902

Environment

VMware vCenter Server 6.x

VMware vCenter Server 7.x
VMware vCenter Server 8.x

Cause

This failure occurs when port TCP 902 is blocked between the source host and the destination host.

Note that cold migrations are considered provisioning traffic and by default go over the management interface. However, if a dedicated provisioning vmkernel interface is configured, it will use this instead.

Resolution

Check for any  firewall rules between the ESXi hosts blocking TCP port 902 and make sure they aren't blocking 902.

 

Refer this document to validate the port connectivity between hosts and vCenter server:

 

Troubleshooting network and TCP/UDP port connectivity issues on ESXi

Additional Information

This issue may also be exhibited while deploying VMs from templates