Cold migrations within the same or different vCenter fail with the error "Cannot connect to host."

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• Migrating a powered-off VM between two hosts within the same vCenter fails at 39%.
• Importing VM between two hosts of different vCenter fails at 39%.
• Log entries in the /var/run/log/hostd.log file of the source ESXi host indicate that the NFC connection to the destination host failed:

info hostd[PID] [Originator@6876 sub=Libs opID=########-#####-auto-####-h5:########-##-##-###### user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] Cnx_Connect: Error message: Failed to connect to server #.#.#.#:902
warning hostd[PID] [Originator@6876 sub=Libs opID=########-#####-auto-####-h5:########-##-##-###### user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect: Failed to connect to server #.#.#.#:902
warning hostd[PID] [Originator@6876 sub=Libs opID=########-#####-auto-####-h5:########-##-##-###### user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect to peer. Error: Failed to connect to server #.#.#.#:902
warning hostd[PID] [Originator@6876 sub=Libs opID=########-#####-auto-####-h5:########-##-##-###### user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] NfcEstablishAuthCnxToServer: Failed to create new AuthD connection: Failed to connect to server #.#.#.#:902
warning hostd[PID] [Originator@6876 sub=Libs opID=########-#####-auto-####-h5:########-##-##-###### user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] [NFC ERROR] Nfc_BindAndEstablishAuthdCnx3: Failed to create a connection with server #.#.#.#:902 Failed to connect to server #.#.#.#:902
error hostd[PID] [Originator@6876 sub=NfcManager opID=########-#####-auto-####-h5:########-##-##-###### user=vpxuser:REDACTED_DOMAIN\REDACTED_USER] Unable to connect to NFC server: Failed to connect to server #.#.#.#:902

Environment

VMware vCenter Server 7.x
VMware vCenter Server 8.x

Cause

This failure occurs when port TCP 902 is blocked between the source host and the destination host.

Note: Cold migrations are considered provisioning traffic and by default go over the management interface. However, if a dedicated provisioning vmkernel interface is configured, it will use this instead.

Resolution

1. Check for any firewall rules between the ESXi hosts blocking TCP port 902 and make sure they are not blocking 902, port connectivity can be checked using the following command:

 

nc -vz <HostIPAddress> 902

 

2. Refer this document to validate further port connectivity issues between hosts and vCenter server:

 

Troubleshooting network and TCP/UDP port connectivity issues on ESXi

Additional Information

This issue may also be exhibited while deploying VMs from templates.