VCF 9.1 deployment fails at "Install VCF Operations and VCF Cloud Proxy using Fleet Lifecycle" due to VCF Operations node FQDN missing from SAN

Products

• VMware SDDC Manager / VCF Installer
• VCF Operations

Issue/Introduction

• VCF Operations 9.1 deployment fails at the Fleet lifecycle OVA component installation with failure at the subtask: Install VCF Operations and VCF Cloud Proxy using Fleet Lifecycle with error message similar to below:
  
  The Fleet lifecycle 'OVA component installation' task with ID <TASK_ID> failed. Internal errors: Failed to deploy VCF Operations node. Check vCenter tasks for errors., Status polling timed out. Review the logs to check operation progress. Increase the timeout value if the operation requires more time., Failed to deploy VCF Operations collector.
  
  
• Generating the VCF Management services log bundle using SOS command /opt/vmware/sddc-support/sos --vcf-mgmt-svcs-logs --zip from SDDC Manager and reviewing the SDDC-build-service logs,  a mismatch in the Subject Alternative Name (SAN) field for one or more of the VCF Operations nodes is observed causing SSL/TLS handshake failure during the first boot validation:
  
  /log/vmware/vcf/sddc-support/sos-<epoch>/VCFMgmtComponents/vcf-management-services/<vcfsr-fqdn>/services-logs/vcf-sddc-lcm/<node-name>/vcf-sddc-build-service-sddcbuild-#########-#####/sddcbuild/console-log-<task_id>.log



  YYYY-MM-DDThh:mm:ss stdout F YYYY-MM-DDThh:mm:ss  WARN sddc-build-service 53 [lcm@4413 scPath="ops" thread="sddc-build-exec-2" logger="c.b.v.t.c.i.SpringTrustStoreReloadInterceptor" method="intercept:84"] [] - SSL/TLS handshake failure for GET https://<VCFOps_Node_FQDN>:443/casa/node/validate/firstboot: Certificate for <VCFOps_Node_FQDN> doesn't match any of the subject alternative names: [<FQDN1>, <FQDN2>, <FQDN3>]
...
YYYY-MM-DDThh:mm:ss stdout F YYYY-MM-DDThh:mm:ss ERROR sddc-build-service 53 [lcm@4413 scPath="ops" thread="sddc-build-exec-2" logger="c.b.v.o.casa.service.OpsCasaService" method="getNodeFirstBootStatus:165"] [] - Failed to get VCF Operations node first boot status
...
YYYY-MM-DDThh:mm:ss stdout F org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://<VCFOps_Node_FQDN>:443/casa/node/validate/firstboot": Certificate for <VCFOps_Node_FQDN> doesn't match any of the subject alternative names: [<FQDN1>, <FQDN2>, <FQDN3>]

• Additionally, the vrops-init.log file from the master node shows that a reverse DNS lookup for the node IP (<VCFOps_Node_IP>) returned an incorrect FQDN instead of the specific node FQDN during initialization:
  
  /opt/vmware/var/log/vrops-init.log

YYYY-MM-DDThh:mm:ss INFO ###[ops@4413 threadId="########" threadName="MainThread"] [root - set_hostname - reverse_lookup] - Reverse lookup got an FQDN for the <VCFOps_Node_IP> IP address: <Incorrect_FQDN_returned_from_DNS>
YYYY-MM-DDThh:mm:ss INFO ###[ops@4413 threadId="########" threadName="MainThread"] [root - set_hostname - set_hostname] - Setting hostname to: <Incorrect_FQDN_returned_from_DNS>
  
  

Environment

VCF 9.1

Cause

A DNS resolution misconfiguration for the node's IP address causes the reverse lookup to return the incorrect FQDN instead of the VCF Operations node's specific FQDN.

 

Resolution

1. Correct the A and PTR records in your DNS server to ensure the VCF Operations node(s) IP(s) point directly to the respective specific node FQDN 

2. Run the following cross-check commands from any node to verify the resolution mismatch has been corrected:

   • Reverse lookup (Must return the node's specific FQDN):
     host -W 15 -R 1 -T <VCFOps_Node_IP>
nslookup <VCFOps_Node_IP>

   • Forward lookup (Must return the node's IP):
     host <VCFOps_Node_FQDN>
nslookup <VCFOps_Node_FQDN>

3. Retry the deployment task from the VCF Installer UI.