Recover management cluster credentials when ~/.kube/config is deleted

Products

• VMware Tanzu Kubernetes Grid

Issue/Introduction

This article explains how to recover management cluster credentials when ~/.kube/config is deleted.

Most Tanzu CLI and Kubectl commands fail if the particular kubeconfig file does not exist or is invalid. In the event that the kubeconfig (~/.kube/config) file has been deleted or corrupted, or the certs become expired, then the following instructions will help in recovering from that problem.

Environment

TKGm 2.0.x+

Resolution

1. ssh into one of the management cluster control plane VMs and become root. You can get the IP address of one of the control plane nodes from vCenter:

ssh capv@CONTROL-PLANE-IP
sudo -i

2. Collect the up to date and working kubeconfig from the management cluster using kubectl: 

kubectl --kubeconfig /etc/kubernetes/admin.conf get secret <Management Cluster Name>-kubeconfig -n tkg-system -o jsonpath='{.data.value}'

3. Decode the data using base64 command and save it to a file on your jumpbox.

$ echo "ENCODED-DATA-FROM-STEP-2" | base64 -d > management-cluster-kubeconfig.conf

4. Update the management cluster kubeconfig with the correct kubeconfig gathered previously. We will update the kubeconfig for the tanzu cli located in ~/.kube-tkg/config. The tanzu cli allows users to define a custom path to the management cluster kubeconfig. In this scenario we should execute "tanzu context list" to see the path to the management cluster kubeconfig, and update this file instead. 

cp management-cluster-kubeconfig.conf /path/to/kubeconfig/admin.conf

5. Finally, we can use the tanzu CLI to get context back for the workload clusters: 

tanzu cluster kubeconfig get CLUSTER-NAME --admin