Update the vCenter Certificate in Telco Cloud Automation (TCA) 2.x
search cancel

Update the vCenter Certificate in Telco Cloud Automation (TCA) 2.x

book

Article ID: 325418

calendar_today

Updated On:

Products

VMware VMware Telco Cloud Automation

Issue/Introduction

Symptoms:
In Telco Cloud Automation (TCA) 2.X, all cluster, nodepool, and Network Function (NF) operations require a secure trust to vCenter via a vSphere certificate thumbprint. 

If a vCenter's certificate has been updated, the following steps must be followed to restore functionality to TCA 2.X

Environment

VMware Telco Cloud Automation 2.3
VMware Telco Cloud Automation 2.0.1
VMware Telco Cloud Automation 2.0
VMware Telco Cloud Automation 2.1
VMware Telco Cloud Automation 2.1.1

Resolution

Update the vCenter thumbprint in TCA and TKG after a vCenter certificate has been updated.

  1. Obtain the vSphere certificate thumbprint.
  2. SSH into each TCA-CP that is attached to the vCenter as the admin user (for minikube).
  3. Run the following command and take note of the Custom Resource (CR) name and namespace: 
    kubectl get VCenterPrime -A 
  4. Run the following command to edit the CR of the target vCenter environment:
    kubectl edit VCenterPrime -n <namespace> <cr_name>
  5. Change the thumbprint field to match the vSphere certificate thumbprint from step 1.
  6. Synchronize the Cell Site Domain Data.
    Note: Wait until the Parent domain's status changes to PROVISIONED before proceeding.
  7. Synchronize the Cell Site Host Data.
    Note: Only resync the host(s) that have failed to provision as resyncing all hosts is a time consuming process.
    Note: Ensure the partial resync checkbox is NOT checked to enable a full resync of the cell site host.

 


For each management cluster:  

  1. SSH into the management cluster control plane VIP with capv

  2. Run the following command and note the CR name and namespace: 

    • kubectl get VCenterPrime –A 

  3. Edit the CR for the target vCenter enviornment with the following command: 

    • kubectl edit VCenterPrime -n <namespace> <cr_name>

  4. Change the "thumbprint" field to match the correct thumbprint from step 1.


If the thumbprint of a market VC changes:

  1. Run the following command and note the CR name and namespace. 

    • kubectl get VCenterSub -A

  2. Edit the CR for the target vCenter enviornment with the following command: 

    • kubectl edit VCenterSub -n <namespace> <cr_name>

  1. Change the "thumbprint" field to match the correct thumbprint from step 1.


Ensure the correct vCenter is being updated by verifying the IP address field in the CR.  

apiVersion: telco.vmware.com/v1alpha1 
kind: VCenterPrime 
metadata: 
  name: vcprime-mgmt-cluster07 
  namespace: tca-system 
spec: 
  server: 
    address: 10.208.70.139  ===> verify the correct vCenter 
    credentialRef: 
      kind: Secret 
      name: vcprime-mgmt-cluster07-secret 
      namespace: tca-system 
  subConfig: 
    datacenter: tcpscale-VMCCloudDC 
    thumbprint: FB:3A:8E:E1:B3:23:DD:FF:F3:6E:19:BE:FE:01:E1:18:E8:24:88:F3 ===> change here 



 

To re-import the TLS thumbprint for TCA-M/TCA-CP (vCenter self-signed certificates only) 

  1. Login to TCA-M/TCA-CP manage page with tca-m/tca-cp:9443 

  1. Re-import the new vCenter TLS thumbprint by “Administration” -> Certificate -> Trusted CA Certificate -> Click the “Import” button  

 

To update the TLS thumbprint for TCA TKG Management Clusters 

 

If vCenter Certificate was changed for a stretch cluster environment:

  1. SSH into management cluster control plane VIP with capv and update the {mgmt-cluster-name}-vsphere-cpi-addon secret in the management cluster context

    • kubectl get secret -A | grep cpi-addon 

  2. Save original CPI vSphere configure to temporary file

    • kubectl get secret -n tkg-system {mgmt-cluster-name}-vsphere-cpi-addon -o jsonpath='{.data.vsphereconf-custom\.lib\.txt}' |base64 -d >/tmp/vsphereconf.txt 

  3. Update the CPI vSphere config with new thumbprint in the temporary file

    • Sample as following of CPI vsphere config: 
       

      [root@tca /home/admin]# vim /tmp/vsphereconf.txt  
((@def vsphere_conf(): -@) 
[Global] 
user = "[email protected]" 
password = "Admin!23" 
port = "443" 
datacenters = "os-test-dc, cellsite-dc" 
[VirtualCenter "10.185.11.97"] 
datacenters = "os-test-dc" 
thumbprint = "13:C1:98:D9:E2:DF:A9:5A:95:4C:6A:96:FA:8D:FE:CF:56:6C:D3:1C" ===> change to new thumbprint 
ip-family = "ipv4" 
[VirtualCenter "sc2-10-185-10-130.eng.vmware.com"] 
datacenters = "cellsite-dc" 
thumbprint = "FD:89:0D:8D:B6:A6:FA:EB:E2:B7:15:CF:D3:F0:57:EB:8C:E3:96:70" 
ip-family = "ipv4" 
[Workspace] 
server = "10.185.11.97" 
datacenter = "os-test-dc" 
thumbprint = "13:C1:98:D9:E2:DF:A9:5A:95:4C:6A:96:FA:8D:FE:CF:56:6C:D3:1C" ===> change to new thumbprint 
ip-family = "ipv4" 

       

  4. Encode the CPI vsphere configure with new thumbprint

    • export encoded_vsphereconf_content=`base64 -w 0 /tmp/vsphereconf.txt` 

  5. Update the secret {mgmt-cluster-name}-vsphere-cpi-addon in tkg-system namespace in the management cluster context, then wait for Kapp reconciliation. Once reconciled, vsphere-cloud-config configmap in kube-system namespace will be updated.

    • kubectl patch secret {mgmt-cluster-name}-vsphere-cpi-addon -n tkg-system -p '{"data": {"vsphereconf-custom.lib.txt":"'${encoded_vsphereconf_content}'"}}' 

  6. Verify the configmap is updated using the below command on the management cluster context:

    • kubectl -n kube-system get cm vsphere-cloud-config -o yaml

  7. Restart the vsphere-cloud-controller-manager pod so that the new configmap is mounted

    • kubectl rollout restart ds/vsphere-cloud-controller-manager -n kube-system 

 

 

 

 

 


Additional Information

 
vSphere TLS Thumbprint may also be included in the vspherecluster CR and in some cases must be updated in both management and all workload cluster contexts. Verify if thumbrpints are being used with the following command:

  • kubectl get vsphereclusters -A | grep -v READY | awk '{print $1,$2}' | while read namespace cluster ; do kubectl get vsphereclusters -n $namespace $cluster -o yaml | grep thumbprint ; done

If vspherecluster CR needs to be updated, use the following steps : 

  1. SSH into management cluster control plane VIP with capv

  2. List all the vsphereclusters including the management cluster:

    • kubectl get vsphereclusters -A
      NAMESPACE    NAME       AGE 
      default-ns      tkg-test-workload   62d 
      default-ns      tkg-wld    83d 
      tkg-system   tkg-mgmt-cluster   83d 

  3. For each cluster, edit the vspherecluster using the below command and update the "spec.thumbprint" field with the correct thumbprint:

    • kubectl edit vsphereclusters -n default-ns tkg-test-workload 

  4. Verify if the update is completed using the below command:

    • kubectl get vsphereclusters -n default-ns tkg-test-workload -o yaml 

For management clusters, use the tkg-system namespace to the kubectl commands:

kubectl edit vsphereclusters -n tkg-system tkg-mgmt-cluster 


Impact/Risks:
Impacts Telco Cloud Automation 2.X

Powered by Wolken Software