HCX - Migration failed with the error: "Unexpected status code: 503"

Products

• VMware HCX

Issue/Introduction

• HCX migrations failed with the error: "Unexpected status code: 503"
• In the /common/logs/admin/app.log, similar error messages are observed:

<timestamps> UTC [###Service_SvcThread-74103, Enterprise: HybridityAdmin, Username: HybridityAdmin, TxId: ] ERROR c.v.v.h.a.vcenter.VcConnection- Error Logging onto VCenter:https://<FQDN-or-IP> User:<username>, message: Unexpected status code: 503

• The /common/logs/admin/app.log file also displays connectivity issues or unresponsive vCenter services prior to the 503 error code:

com.vmware.vchs.hybridity.adapters.vsphere.vim.VsphereServiceUnavailableException: Error communicating to VC endpoint <FQDN-or-IP>:443. Reason: Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x00007fecf8009ca0] _serverNamespace = /sdk action = Allow _port = 8085)

Cause

If vCenter becomes unresponsive during an ongoing migration or if vCenter is upgraded during scheduled HCX migration, this can disrupt the migration process, as HCX depends on vCenter services.

Resolution

We do not recommend rebooting HCX managers, vCenter server, or the source and target cluster hosts while migrations are in progress. Multiple backend jobs run in the background, and restarting any of these components or their services may disrupt the migrations.

Workaround:
Restart the migration job that failed.

Additional Information

When using Bulk or RAV (Replication Assisted vMotion) migration, enable 'Seed Checkpoint'. In the event that a migration is unsuccessful or canceled, Seed Checkpoint retains the target disks created at the target site. Without Seed Checkpoint, the HCX roll back process cleans up the target disks created during the migration and all transferred data is lost. 

For more information, please access: HCX - Health Check and Best Practices