'Configure NSX Fabric Compute Manager Task' fails in VCF 9.0 deployment due to a vCenter SSL THUMBPRINTS MISMATCH

Products

• VMware SDDC Manager

Issue/Introduction

Symptoms:

• After applying a Scenario 1 resolution about 'Configure NSX Fabric Compute Manager Task' fails in VCF 9.0 deployment due to an incomplete vCenter machine SSL certificate chain, the customer is facing the following issue:
  
  domainmanager.log
YYYY-MM-DDThh:mm:ss.694+0000 ERROR [vcf_dm,69######################,####] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-28]  [QUJCDG] VCF_SSL_THUMBPRINTS_MISMATCH Could not establish trust with pre-existing product vcenter.example.com. Provided SSL thumbprint: 3B:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX, actual SSL thumbprint: 2E:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Could not establish trust with pre-existing product vcenter.coexya.lan. Provided SSL thumbprint: 3B:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX, actual SSL thumbprint: 2E:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX
  
  

 

Environment

VMware Cloud Foundation (VCF) 9.0
vCenter Server
NSX-T

Cause

The issue is caused by a mismatch between a new vCenter SSL thumbprint and an old SSL thumbprint already present in the workflow.json file.
 

Resolution

This Workaround needs to be followed while not initiating the installer wizard from scratch

• SSH to the VCF Installer appliance with vcf user and elevate to root with su
  Run the below command to add a parameter in domainmanager application-prod.properties file and restart domainmanager service
  echo vcf.vault.http-access=true >> /etc/vmware/vcf/domainmanager/application.properties && systemctl restart domainmanager
   
• Wait a minute or so for the domainmanager service to restart
• Run the below command to copy the workflow.json
  curl -X GET http://localhost/domainmanager/internal/vault/<execution ID identified from domainmanager.log> | json_pp > /tmp/workflow.json
   Sample
  curl -X GET http://localhost/domainmanager/internal/vault/86ba####-####-####-####-########5eb7 | json_pp > /tmp/workflow.json
  
  
• SSH to the vCenter wit a root account and run the following command to confirm the new vCenter SSL thumbprint:
  echo | openssl s_client -connect vcenter-fqdn:443 -showcerts | openssl x509 -fingerprint -noout
   
• Edit /tmp/workflow.json file and replace "sslThumbprint": 'old vCenter SSL thumbprint' with 'new vCenter SSL thumbprint'
  Update the modified workflow.json in VCF Installer
  curl -H 'Content-Type:text/plain' -X PUT http://localhost/domainmanager/internal/vault/86ba####-####-####-####-########5eb7 -d @/tmp/workflow.json
   
• Retry the failed run from the VCF Installer UI

Additional Information

Re-try an existing workflow by modifying the workflow spec file
https://knowledge.broadcom.com/external/article/314620/retry-an-existing-workflow-by-modifying.html