vSphere with Tanzu Deactivation Fails with "Cleanup requests to NSX Manager Failed"

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

^{When attempting to delete a Supervisor cluster, the process gets stuck because the deletion of NSX objects from NSX Manager fails. This causes the cleanup request to fail, resulting in the supervisor cluster being unable to delete successfully.} 

^{/var/log/vmware/wcp/wcpsvc.log:}

NSX resource removal did not fully complete for zone=######### in cluster=domain-c8. NSX cleanup failed: exit status 1.
Unable to disable cluster domain-c8. Err NSX cleanup failed: NSX cleanup failed: exit status 1. 

Environment

^{VMware vSphere with Tanzu}

Cause

^{The issue occurs when the NSX resources associated with the Supervisor cluster cannot be fully removed. The failure is due to an unsuccessful NSX cleanup process.} 

Resolution

^{To resolve the issue, perform the following steps:}

1. ^{Backup the Cleanup Script: First, back up the existing cleanup script to avoid any loss of the original file.}

# cp /usr/lib/vmware-wcp/nsx_policy_cleanup.py  /usr/lib/vmware-wcp/nsx_policy_cleanup.py.bk

2. ^{Edit the Cleanup Script: Modify the cleanup script to ensure the correct NSX Manager credentials and configuration are provided.}

• Edit the Script

# vi /usr/lib/vmware-wcp/nsx_policy_cleanup.py

• ^{Modify the following parameters in the script:}

self.username = "nsxusername"  # NSX Manager username
self.password = "nsxpassword"  # NSX Manager password
self.vc.endpoint = None        # Set the vCenter endpoint to None
self.vc_username = None        # Remove vCenter username
self.vc_password = None        # Remove vCenter password

• ^{Save and exit the file by typing :wq!.}

3. ^{Restart the WCP Service: Restart the WCP service to apply the changes made in the cleanup script.}

# vmon-cli -r wcp

4. ^{Run the Cleanup Script: Execute the cleanup script manually to remove the NSX resources associated with the Supervisor cluster.}

# python3 /usr/lib/vmware-wcp/nsx_policy_cleanup.py --cluster domain-c8:########### -u admin -p 'nsxmangerpassword' --mgr-ip=##.##.##.## --no-warning --top-tier-router-id=domain-c8:########## --all-res -r

^{Ensure you replace the placeholders above with the appropriate values:}

• ########### - The specific identifier for your cluster.

• nsxmangerpassword - Password for the NSX Manager.

• ##.##.##.## - IP address of your NSX Manager.

• ########## - Top-tier router ID.

^{After performing these steps, the Supervisor cluster should be able to delete successfully, and the NSX Manager cleanup issue will be resolved.}