Update vCenter certificate for TKG cluster

Products

• VMware Telco Cloud Automation

Issue/Introduction

All clusters, nodepools, and Network Function (NF) operations require a secure trust to vCenter via a vSphere certificate thumbprint. 

Environment

3.0, 3.1, 3.2

Cause

If a vCenter's certificate has been updated, the vCenter certificate and thumbprint must be updated to restore functionality

Resolution

Resolved in TCA 3.3. See Updating Certificate/user Credentials of vCenter on TCA and Linked CaaS Clusters

Workaround:

1. Re-Import the vCenter Certificate to TCA-M and TCA-CP

• Log in to the TCA Appliance Manager 
  <tca-m/tca-cp-ip>:9443.
  
  
• Click Certificate > Trusted CA Certificate > IMPORT.
  
  
• Select the trusted certificate type that you want to import and do one of the following:

• • • Browse and select the file to import.
    • Type the URL of the certificate.
    • Paste the certificate file content.
      
      
• Click Apply.

     2. Update the vCenter thumbprint in the TKG Cluster(s 

• SSH into the TCA-CP that controls the management clusters using the admin credentials.
  ssh admin@<tca-cp-ip>
  
  
• Check connectivity to retrieve the update-vc-tp script.

curl -kfsSL https://packages.broadcom.com/artifactory/tca-distro/kb/vc-updater/tca3.0/update-vc-tp.sh | bash -s -- -h

NOTE: For airgap environments, users should download the script to another location first, then copy it to the TCA-CP.

• Run the script with the vCenter IP
  
  curl -kfsSL https://packages.broadcom.com/artifactory/tca-distro/kb/vc-updater/tca3.0/update-vc-tp.sh | bash -s -- -d <vCenter-ip>
  
  NOTE: In case of different vCenter for management clusters and for Workload clusters, execute the above command by giving the both vcenter (first execute the command with the VC address responsible for MC and then the same command with WC vCenter ip)
  
  
• Navigate to Connected Endpoints, find the corresponding vCenter, and confirm the status of the vCenter has been modified.

• Click and dismiss the message to acknowledge the vCenter certificate change.
  
  
• In TCA-M UI, Validate the vCenter credentials in the virtual infrastructure and save the changes.

1.  

Additional Information

In some cases the script may report vSphere CSI Secret Thumbprint mismatch for workload clusters. In this case vSphere-csi daemon set pods go into CrashLoopBackOff state after vCenter thumbprint is updated. Or, the vsphere-csi pods could take several minutes to be recreated.

Cluster Name                      : worload-cluster1
        Cluster Type              : classy
        TKG VSphere CSI Secret    :
                Thumbprint        : <REDACTED>   Mismatched.
        TKG VSphere CPI Secret    :
                Thumbprint        : <REDACTED>   Mismatched.

Workaround:

For clusters with node pools, recreate the vsphere-csi package installation by running below commands on the workload clusters:

# For classy clusters kubectl delete pkgi -n tkg-system <cluster-name>-vsphere-csi

# For standard clusters kubectl delete pkgi -n tkg-system vsphere-csi

For clusters without node pools, restart the vsphere-csi-controller pod by running below command on the workload clusters:

kubectl rollout restart deploy/vsphere-csi-controller -n vmware-system-csi

Note: This workaround is added to TCA 3.3