VCF Operations upgrade to 9.1 fails with BROWNFIELD_VCF_OPS_NODE_CERTIFICATE_VALIDATION.error due to missing SAN entries

Products

• VCF Operations

Issue/Introduction

• During a VMware Cloud Foundation (VCF) upgrade from 9.0.x to 9.1, the "Deploy VCF Management Components" or "Import VCF Operations in Fleet Lifecycle" task fails.
• The /var/log/vmware/vcf/domainmanager/domainmanager.logof the SDDC Manager reports the following error:
  
  "errorCode": "BROWNFIELD_VCF_OPS_NODE_CERTIFICATE_VALIDATION.error" "message": "Unable to reach VCF Operations node [FQDN]. This could indicate a network connectivity issue or a certificate that does not include this address in its SAN field"

Environment

• VMware Cloud Foundation 9.0.x
• VMware Cloud Foundation 9.1
• VCF Operations

Cause

The VCF 9.1 upgrade process performs strict validation of the VCF Operations node certificates. If the existing certificate does not explicitly include the FQDN, IP address, and shortname of all nodes in the VCF Operations cluster within the Subject Alternative Name (SAN) field, the validation task fails, preventing the upgrade from proceeding.

Resolution

To resolve this issue, the VCF Operations certificate must be regenerated to include the missing SAN attributes for all cluster nodes.

Procedure

1. Review Requirements: Refer to the official Broadcom documentation on configuring certificates for VCF components: Configure a certificate for use with VCF.
2. Generate Certificate Signing Request (CSR): When regenerating the certificate for the VCF Operations nodes, ensure the CSR includes the following SAN entries for all nodes in the cluster:
   • Fully Qualified Domain Names (FQDNs)
   • IP Addresses
   • Shortnames (hostnames)
3. Apply Certificate: Follow the steps in the documentation linked above to replace the certificate on the VCF Operations nodes.
4. Validate: After replacing the certificate, verify that the new certificate contains the required SAN entries by running the following command against each node: echo | openssl s_client -showcerts -connect [FQDN]:443 2>/dev/null | openssl x509 -noout -text | grep -A1 "Subject Alternative Name"
5. Retry Task: Once validated, navigate to the SDDC Manager UI and retry the failed upgrade task.

Additional Information

If the upgrade task continues to fail after certificate rotation due to stale database records, ensure that you have cleared invalid metadata from the Fleet Lifecycle management database using the remediation script provided in KB 441858.