Management cluster appears healthy in the UI, but is 'Deleting' in CLI

Products

• VMware Telco Cloud Automation

Issue/Introduction

Using 'kbsctl show managementclusters' or ccli > show mc, the status of the management cluster is in Deleted State:

Count: 1
----------------------------------------
ID: <MGMT_CLUSTER_UUID>
Name: management-cluster
Status: Deleted
TKG ID: <UUID>

Using 'kubectl get namespace' from the TCA-CP minikube context does not show the management cluster namespace.

In the TCA-M GUI, the management cluster is shown as Green and Active.

Environment

2.3

Cause

Manual deletions of a cluster's namespace or cluster resources can result in inconsistencies between the CLI and the GUI.

Resolution

To restore the namespace to minikube, a backup of minikube will be necessary. If one exists, it should be in '/opt/vmware/k8s-bootstrapper/'

kubectl apply -f /opt/vmware/k8s-bootstrapper/minikubecr.backup-<YYYYMMDD-HHMMSS> --kubeconfig=/home/admin/.kube/config

Confirm the namespace has been restored on the TCA-CP:

kubectl config use-context minikube

kubectl get namespace

 

To reset the management cluster status to 'Running' in CCLI perform the steps below-

SSH to TCA-CP.
Switch user to root

su -i

Run the command below to reset the management cluster status          

kbsctl debug set-cluster-status -i <MGMT_CLUSTER_UUID> -s Running 

This will reset management cluster status to 'Running' in CCLI as well.