vSphere namespace stuck in removing state in vCenter UI

Products

• VMware vSphere with Tanzu
• vSphere with Tanzu

Issue/Introduction

The process of removing the namespace in the vsphere environment has not been completed successfully, resulting in terminating state in the supervisor cluster and removing state in the vcenter. 

Cause

This situation arises due to multiple reasons; a few are listed below.

- The namespace may have resources or dependencies that are preventing it from being removed.

- There might be an issue with the API or configuration that is causing the namespace removal process to hang.

- Sometimes, the namespace might be in a stale or corrupted state that prevents proper deletion.

- Network or connectivity issues between your management tools and the vSphere environment could cause the deletion process to hang.

Resolution

- Check if there are any remaining resources  associated with the namespace. Ensure that all resources have been deleted or reassigned before attempting to remove the namespace again.

kubectl get all -n <namespace>

- describe the namespace and check the event to see why it is stuck in a terminating state.

kubectl describe <namespace>

- Delete the offending service from supervisor VM, if it doesn't work run below script and figure out any remaining resources under the namespace and delete them manually.

for resource in `kubectl api-resources | grep -v NAME | awk '{print $1}'`; do echo $resource; kubectl get $resource -A 2> /dev/null | grep <namespace> ; done

- Post deleting the associated resources,  delete the namespace and restart the WCP service.