TCA 2.3 - CaaS TKG cluster is in inconsistent state after cluster backed VMs deleted from vCenter UI

Products

• VMware
• VMware Telco Cloud Automation

Issue/Introduction

Symptoms:
- Unable to delete the Management and workload cluster from TCA UI as the VIM registration is not unregistered gracefully
- When we try to unregister Harbor from Partner system we get the following error:
 

error while trying to de-register a cluster from Harbor.user
confirmed that password is correct getting error Login to Remote Cloud failed with errors :
[{"code":"E_OPERATION_FAILED","message":"Login to cloud with provided credentials failed"}]

- Cannot delete the inconsistent Network Functions . Error: 
 

This VIM is used by some of the network function instances. 
First you must delete all the instances deployed on this VIM and then try to delete the VIM.
Please contact the system administrator.

Environment

VMware Telco Cloud Automation 2.3

Cause

To de-comission / delete the CaaS cluster from TCA UI first which in turn deletes the backend VMs from vCenter UI and not vice versa.

Resolution

Use the attached script and sql file to cleanup the system in TCA 2.3
 

Usage :

./cleanExtensionForVimName.sh <workload_cluster_name>

Example:

./cleanExtensionForVimName.sh wld-ag-lab-migrate-v1

Steps to execute the script :

1. Download the attached scripts cleanExtensionForVimName.sh & FilterVimNameFromExtension.sql 
Copy to TCA-M appliance /home/admin/ folder. Note that both files have to exist in the same folder.

2. SSH to tca appliance as ‘admin’ using appropriate credentials.

3. Make the scripts executable [chmod +x filename] then run the script using below command:

./cleanExtensionForVimName.sh workload_cluster_name

4. Verify via TCA UI --> Partner Systems UI that the cluster VIM that was included in the above script is not listed under ‘VIM Associations’ for any of the registered Harbor systems.

5. From the TCA partner systems UI, click on 'Edit' button adjacent to harbor that was de-registered. Click on next after providing the harbor password on next screen. Confirm that other workload cluster(s) that are associated with the harbor are selected from the map view and hit finish.

6. Confirm that there are no network functions deployed on the cluster that is to be deleted.

7. Delete the cluster from ‘Virtual Infrastructure’ page of TCA UI after filtering out and identifying the cluster to be deleted. Verify that the delete goes through and doesn’t throw any error indicating cluster is associated with one or more repositories.

Note that the VIM delete will work only if there are no NF’s instantiated on this cluster via TCA UI.

Orphaned NFs may need to removed first so that VIM can be deleted even after partner repository de-registration is done via attached script.

 

 

Additional Information

Impact/Risks:
CaaS cluster inconsistent and cannot redeploy or re-instantiate the CNF