All kubectl commands in TKC failed due to VIP mismatch in NSX-ALB environment

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

Symptom:

• Run any commands in TKC context failed with error " couldn't get current server API group list: Get "https://<correct-IP>:6443/api?timeout=32s": dial tcp <correct-IP>:6443: connect: connection refused ".
• In NSX-ALB UI - Virtual Services, found virtual service associated with this TKC has another different IP address as well as  "kubectl get svc -n <tkc-namespace>" in Supervisor context.

 

 

Environment

VMware vSphere Kubernetes Service with VC version below 8.0u3e

Cause

Please refer to cause section in How to recover supervisor and guest cluster control plane LB IP hosted on Avi Load balancer

Resolution

Refer to KB 390201 Resolution to correct IP address mapping
How to recover supervisor and guest cluster control plane LB IP hosted on Avi Load balancer

The issue is fixed in VC 8.0u3e and above.