Error: "Failed to deploy VCF Operations fleet management" during VMware Cloud Foundation 9.x deployment

Products

• VMware Cloud Foundation

Issue/Introduction

• During the deployment of VMware Cloud Foundation (VCF) 9.0.0, the process fails at the Fleet Manager deployment stage. This issue prevents the orchestration of SDDC Manager tasks and halts the environment creation.

• Within the VCF Installer UI, you see the following error:

  Failed to deploy VCF Operations fleet management on SDDC Manager Reference Token: ######

• When the /var/log/vmware/vcf/domainmanager/domainmanager.log file on the SDDC Manager appliance is reviewed, a VCF_ERROR_INTERNAL_SERVER_ERROR is observed along with the following TLS and certificate errors:

  • certificate_unknown(46)

  • Unable to construct a valid chain

  • Unable to find certificate chain

Environment

VMware Cloud Foundation 9.x

Cause

This issue occurs because the SDDC Manager appliance does not trust the newly generated Fleet Manager certificate. When the SDDC Manager attempts an API invocation to the Fleet Manager, it cannot construct a valid certificate chain or verify the certificate, resulting in an SSL handshake failure (certificate_unknown).

Resolution

To resolve this validation failure, manually extract the certificate from the Fleet Manager appliance and import it into both the commonsvcs and Java trust stores on the SDDC Manager appliance.

1. Log in to the SDDC Manager appliance via SSH using the vcf user account.

2. Switch to the root user:

   su - root
   

3. Retrieve and export the certificate from the Fleet Manager to the SDDC appliance. (Replace <Fleet_Manager-FQDN> with the actual Fully Qualified Domain Name or IP address of your Fleet Manager):

   openssl s_client -connect <Fleet_Manager-FQDN>:443 -showcerts </dev/null 2>/dev/null | openssl x509 -outform PEM > /tmp/fleet_manager.crt
   

4. Obtain the SDDC trusted certificates key:

   KEY=$(cat /etc/vmware/vcf/commonsvcs/trusted_certificates.key)
   

5. Import the exported Fleet Manager certificate into the commonsvcs trust store. Type yes when prompted to trust the certificate.

   keytool -importcert -alias fleet -file /tmp/fleet_manager.crt -keystore /etc/vmware/vcf/commonsvcs/trusted_certificates.store --storepass $KEY
   

6. Import the certificate into the Java trust store. Type yes when prompted to trust the certificate:

   keytool -importcert -alias fleet -file /tmp/fleet_manager.crt -keystore /etc/alternatives/jre/lib/security/cacerts --storepass changeit
   

7. Restart the SDDC Manager services to apply the new trust store configurations:

   /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh
   

8. Wait for all services to fully restart and report as healthy.

9. Return to the VCF Installer UI and retry the deployment.