Stale NSX Networks Remain After Deleting a VCF Cluster or Workload Domain

Products

• VMware SDDC Manager

Issue/Introduction

When deleting a cluster or Workload Domain (WLD) in VMware Cloud Foundation (VCF), stale network data may be left behind in VMware NSX. This prevents the complete automated removal of the environment and necessitates complex manual cleanup of the orphaned NSX components.

Environment

VMware Cloud Foundation (VCF)
VMware NSX
VMware Aria Operations for Networks (vRealize Network Insight / vRNI)

Cause

This issue occurs due to two distinct software logic conditions encountered during the deletion workflow:

1. Aria Operations for Networks (vRNI) latency and statistics collection alters the ownership metadata of the networks. This marks them as not being owned by NSX, which consequently revokes NSX's permissions to automatically remove them.

2. The presence of any Virtual Machines (VMs) in the target cluster or WLD triggers a safety bypass that halts network deletion. While designed to protect essential management VMs from accidental deletion, this bypass incorrectly executes even when the target is strictly a workload (non-management) WLD.

Resolution

To prevent stale data and ensure a clean deletion, the following preparatory steps must be completed before initiating the deletion of a cluster or Workload Domain:

1. Disable and remove all latency and statistics collection (associated with vRNI) from the target networks.

2. Power off and delete (or migrate) all Virtual Machines located within the target cluster or Workload Domain.

Completing these steps clears the conditions that trigger the bypass logic, allowing the VCF automated workflows to successfully remove the associated NSX networks.

Additional Information

Internal engineering tickets have been opened to address the underlying logic and bypass conditions in a future release.

If a cluster or WLD has already been deleted without these preparatory steps, manual cleanup of the stale NSX objects via the NSX Manager API or UI is required.

Manual cleanup after the fact can be done by following "Attempting to Remove NSX for an ESXi host Transport Node in NSX Manager fails when the host is not available in vSphere." and "Stale host entry removal" and opening a support ticket with Broadcom if assistance is needed.