How to manually synchronize renewed CaaS kubeconfig for VMware Telco Cloud Automation 2.3
search cancel

How to manually synchronize renewed CaaS kubeconfig for VMware Telco Cloud Automation 2.3

book

Article ID: 371488

calendar_today

Updated On:

Products

VMware Telco Cloud Automation

Issue/Introduction

Users notice that their K8S VIMs are Disconnected in TCA-M, or the status of the Workload Cluster is disconnected (indicated with a red dot) in TCA-CP Appliance Management portal (9443) or the Harbor is in initiated state.

The harbor connection status will be in Initiated state as shown in below snippet


TCA web.log will show below errors

2024-10-01 10:56:34.245 UTC [https-jsse-nio-127.0.0.1-8443-exec-5, Ent: HybridityAdmin, Usr: [email protected], TxId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] INFO c.v. vca.hybridity.util. NSPRestClient- Token last updated 14+ minutes ago, refreshing token ..
2024-10-01 10:56:34.316 UTC [https-jsse-nio-127.0.0.1-8443-exec-5, Ent: HybridityAdmin, Usr: [email protected], TxId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] WARN c
c.v.vca. hybridity. util. NSPRestClient- Login to cloud https://<TCACP FQDN> failed, with status 401:
Usr: [email protected], TxId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] ERROR com. vmware. hybridity.nfvm. VnfUtils- Failed to query the TCA-CP Api POST:/hybridity/api//repositories/query.
java. lang.SecurityException: Login to cloud https://<TCACP FQDN> failed, with status 401:
2024-10-01 10:56:40.296 UTC [https-jsse-nio-127.0.0.1-8443-exec-8, Ent: HybridityAdmin, Usr: [email protected], TxId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] WARN c.v.vca.hybridity.util.NSPRestClient- Login to cloud https://<TCACP FQDN> failed, with status 401:

 

 

Environment

VMware Telco Cloud Automation 2.3, 2.3.0.1

Cause

Starting from TCA 2.3, TCA supports automatic certificate renewal for both management cluster and v2 workload clusters. Refer this for more details.

In certain cases, the certificates are renewed, but these are not synced to the TCA appliances. In such scenarios, users would notice that the relevant K8S VIM is Disconnected in TCA-M, or the status of this Cluster might be disconnected (indicated with a red dot) in TCA-CP Appliance Management portal (9443).

In such cases, it is required to manually update the Cluster certificates and / or kubeconfig stored in TCA database.

There are 2 parts to updating the CaaS Cluster certificates:

  1. Updating the Cluster certificate within the Cluster itself
  2. Updating the references of new Cluster Certificates within TCA-M and TCA-CP

This KB talks about step 2.

The assumption here is that the individual Cluster certificates have all been replaced correctly. If not, please follow the entire KB here: https://knowledge.broadcom.com/external/article?legacyId=94761

Resolution

This has been resolved in VMware Telco Cloud Automation 2.3.0.2 and newer versions (3.x).

For a manual resolution for VMware Telco Cloud Automation 2.3 or 2.3.0.1, please follow the steps below.

 

Synchronise the updated kubeconfig  within TCA-M and TCA-CP

Note: All (upgraded and non-upgraded) Clusters require the kubeconfig to be synchronised

i.e. both upgraded and non-upgraded Clusters should follow this procedure.

 

  1. First check the status of the Management Cluster within TCA-CP Appliance Management portal (9443). If this is red, then execute the steps below (including point 7) for the Management Cluster first, and then continue with the steps 2 - 7 for the Workload Clusters.

  2. POST the following API call, from any machine that has access to the TCA-M web layer, to generate an authentication token:

    curl -D - --location --insecure --request POST 'https://tca-m-url/hybridity/api/sessions' --header 'Accept: application/json' --header 'Content-Type: text/plain' --data-raw '{"username": "username","password": "plain_text_password"}'


    Note: Replace tca-m-url and username and plain_text_password with the actual values in the command provided.

  3. Take note of the x-hm-authorization from the output of the previous step:

    Sample: 95XXXXX4:dXX2:4XX3:bXX2:7XXXXXXXXXX5

  4. Update the TCA-M and TCA-CP database by synchronising the kubeconfig:

    curl --location --insecure --request POST 'https://tca-m-fqdn/telco/api/caas/v2/clusters/cluster_name/syncKubeconfig' --header 'Accept: application/json' --header 'Content-Type: application/json' --header 'x-hm-authorization: auth-token'


    Note: Replace tca-m-fqdn and cluster_name and auth-token with the actual values in the command provided.
    Note: The operation can take several minutes, make a note of job_id received in the response which requires in the next step.

  5. To ensure that the operation is succeeded, run the following API call:

    curl --location --insecure --request GET 'https://tca-m-fqdn/hybridity/api/jobs/job_id_from_above_response' --header 'Accept: application/json' --header 'x-hm-authorization: auth-token'


    Note: Replace tca-m-fqdn and auth-token with the actual values in the command provided.
    Note: Take note of the isDone and didFail flags. The isDone flag should return true and the didFailflag should return false.

  6. SSH login to TCA-CP to restart the services:

    ssh admin@tca-cp
su -


    Note: tca-cp where the cluster is configured
  7. Restart the following TCA-CP services:

    systemctl restart app-engine
systemctl restart web-engine

    Note: In case of multiple TCA-CPs (i.e one for Mgmt cluster & one for Workload cluster) the app & web services should be started from both.

Powered by Wolken Software