VCF Management 9.1 Deployment Fails at 'Import VCF Operations in Fleet Lifecycle' step

Products

• VCF Operations

Issue/Introduction

During a VMware Cloud Foundation (VCF) 9.1 upgrade or deployment, the VCF Management deployment process fails and halts at the subtask: Import VCF Operations in Fleet Lifecycle.

In the deployment UI, the subtask under the Deploy VCF Management Components workflow shows a Failed status.

 

On the VCF Installer logs /var/log/vmware/vcf/domainmanager/domainmanager.log we see connectivity issues: 

 INFO  [vcf_dm,##############################] [c.v.e.s.c.s.c.CommonsServicesSdkClient,http-nio-<IP>-####-exec-1]  Common Services API Client connection timeout is 30000 ms
 DEBUG [vcf_dm,##############################] [c.v.v.d.notif.impl.pc.PcPollingTask,vcf-ds-<task-ID>]  [Sub Id: '<SUB-ID>', Data source id: '<Datasource-ID>'] Timeout while waiting for updates. Retrying with version '1'

[c.v.v.v.s.VspClusterServiceImpl,http-nio-<IP>-####-exec-1]  Could not retrieve the VSP component details
com.vmware.vsp.invoker.ApiException: Message: java.net.NoRouteToHostException: No route to host
HTTP response code: 0
HTTP response body: null
HTTP response headers: null



ERROR [vcf_dm,##############################] [c.v.v.v.s.fleetlcm.FleetLcmService,dm-exec-###]  Invalid SddcSpec state - can't fetch/determine Fleet LCM address!

 

Network connectivity has been verified and able to ping between SDDC (Software-Defined Data Center) Manager and the VCF Service platform FQDN (the Fully Qualified Domain Name of the management VSP cluster)
Note: We can do this by SSHing into the SDDC Manager and pinging the FQDN

 

Cause

This issue is typically caused if not by network connectivity issues by expired, out-of-sync, or locked management credentials on the SDDC Manager.

When the Fleet Lifecycle import process attempts to establish a secure connection between the new VCF Operations environment and the SDDC Manager, the internal service account authentication fails. This results in a connectivity or authorization exception, such as a NoRouteToHostException or a service account rejection, because the stored credentials used to bridge the components are invalid or mismatched following the initial phases of the 9.1 upgrade orchestration.

Resolution

To resolve this issue, perform a password rotation for the affected management accounts in SDDC Manager to re-sync the authentication vault.

1. Reference How to reset SDDC Manager vcf and root user accounts to replace the password.

 

Forcing a fresh password update through the SDDC Manager Password Management framework overwrites stale or locked internal credentials across the ecosystem. This allows the Fleet Lifecycle subtask to successfully complete the security handshake and clear the internal authentication exceptions.