VCF Automation 9.0.x to 9.1 upgrade precheck failure during certificate chain validation

Products

• VCF Automation

Issue/Introduction

Upon attempting to import VCF Automation 9.0.x into a VCF 9.1 environment, prechecks fail due to a certificate chain validation failure.

Environment

• VCF Automation 9.0.x
• VCF Automation 9.1.0
• VCF Services Runtime 9.1.0

Cause

• The Import workflow extracts the certificate from the vmsp-tls secret on the source (VCF Automation 9.0.x) cluster to produce a data dump that is used during prechecks.
• If the tls.crt property of the vmsp-tls secret in the instio-ingress namespace does not contain the full chain, including intermediate CAs then the certificate validation precheck will fail.

Resolution

Note: Before performing any maintenance activities, such as upgrades, it is recommended to take an on-demand backup of the VCF Automation.

1. Manually update the vmsp-tls secret in the istio-ingress namespace on the VCF Automation 9.0.x cluster using kubectl. Prepare the full chain certificate in a file. It should include the leaf cert, intermediate CA and root CA in sequence.
2. Prepare a file that contains the PEM formatted certificates:
   

   leaf
   intermediate CA
   root CA

3. Run the following command while SSH into one of the VCFA 9.0.x nodes:
   

   ssh vmware-system-user@<vcf-a node>
   sudo su -
   export KUBECONFIG=/etc/kubeconfig/admin.conf
   
   FULL_CHAIN=$(cat full-chain.pem)
   
   kubectl patch secret vmsp-tls -n istio-ingress -p "{\"data\":{\"tls.crt\":\"$(echo -n "$FULL_CHAIN" | base64 -w 0)\"}}"

4. Go back to the LCM UI and click on the ellipsis (...) menu beside UPGRADE, and choose Run Prechecks.