How to remove stale Global Manager entries remain visible in UI and cannot be deleted

Products

• VMware NSX

Issue/Introduction

In VMware NSX environments, users may encounter a scenario where a stale Global NSX Manager entry remains visible in the user interface and cannot be deleted. The Delete option for the stale node is greyed out.

Symptoms:

• vCenter server certificate has been replaced.
• A decommissioned or removed Global Manager appliance persists in the NSX inventory.
• NSX Global Managers reflect a "REPO_SYNC" status as "FAILED" or "IN PROGRESS".
• Attempts to remove the orphaned node via the UI fail or the option is unavailable.
• Attempt to remove the orphaned node via CLI using the below command gives the below error.
  curl -k -u admin -X POST "https://<NSX_Manager_IP>/api/v1/cluster/<STALE_NODE_UUID>?action=remove_node"

root@nsx-mgr-01:~# curl -k -u admin -X POST "https://192.##.##.##/api/v1/cluster/d5####-5f##-11##-98##-00##########?action=remove_node"
Enter host password for user 'admin':
{
  "details" : null,
  "error_code" : 36754,
  "module_name" : "node-services",
  "error_message" : "[ClusterBootManager] Cannot detach d5####-5f##-11##-98##-00########## node as the repository is not synchronized on other nodes. Please make sure that repository is synchronized on at least one of the other node. You can check the synchronization status on 'System'-->'Appliances' page under 'VIEW DETAILS' tab.",
  "error_data" : null,
  "related_errors" : null

Environment

VMware NSX
VMware NSX Global Manager

Cause

The issue is caused by a communication breakdown between the NSX Global Manager and vCenter Server, after a vCenter certificate replacement.

If a Global Manager appliance is removed from vCenter while the connectivity/trust between NSX Global Manager and vCenter is disrupted, the automated inventory cleanup process in NSX fails. This leaves an orphaned inventory record in the NSX Global Manager database that cannot be reconciled through standard UI workflows until synchronization is restored.

Resolution

To resolve this issue, you must first restore the management plane communication before manually purging the stale entry via API.

Prerequisites

1. Perform a backup of the NSX Manager cluster.
2. Ensure you have admin credentials for the NSX Manager.

Step 1: Restore Compute Manager Connectivity

1. Log in to the NSX Manager UI.
2. Navigate to System > Fabric > Compute Managers.
3. Select the vCenter Server associated with the stale entry and click Edit.
4. Re-enter the credentials if prompted and click Save.
5. Accept the new vCenter certificate thumbprint to restore the connection status to Up.

Step 2: Synchronize Global Manager Inventory

1. Navigate to System > Configuration > Global Manager Settings.
2. Ensure that the synchronization status is updated. If the managers remain "REPO_SYNC" state is "IN PROGRESS" wait for the background reconciliation to complete now that the Compute Manager is connected.

Step 3: Remove the Stale Node via API

Once connectivity and synchronization are verified, use a REST API client (such as curl) to remove the orphaned node from the cluster configuration.

1. Identify the UUID of the stale Global Manager node.
2. Run the following command from a terminal with access to the NSX Manager:
   
   curl -k -u admin -X POST "https://<NSX_Manager_IP>/api/v1/cluster/<STALE_NODE_UUID>?action=remove_node"

 

Note: Replace <NSX_Manager_IP> with your Manager's FQDN or IP, and <STALE_NODE_UUID> with the ID of the orphaned node.

Step 4: Verification

1. Refresh the NSX Manager UI.
2. Verify that the stale Global Manager entry is no longer visible in the inventory or cluster status pages.