Supervisor cluster stuck at Supervisor deleted

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• The supervisor is stuck in Deleting state as follows:
• No EAM Agents are present
  
  
• No supervisor control plane/worker nodes or VMs are present.
  
  
• Running "/usr/lib/vmware-wcp/decryptK8Pwd.py" gives an output, however we cannot SSH since the CP VMs do not exist.
  
  
• NSX cleanup starts but does not report a completed status.

Environment

vSphere Kubernetes Service

Cause

The process to delete and clean up the supervisor got permanently stuck during the networking removal phase because a safety timer failed to work.

Resolution

Ensure that a manual NSX cleanup completes or returns 0 resources to clean up:
Refer: VKS Supervisor Fails to Deactivate due to error "Cleanup requests to NSX Manager Failed"

Reach out to VMware by Broadcom Technical Support for assistance as the next steps involve editing the vCenter database directly.

Additional Information

If the Namespace resource pool and supervisor folder are still present in the inventory, reach out to Broadcom Support for cleaning up the stale resources.