Segment fails to delete from Global Manager. Status is "Delete in Progress"

Products

• VMware NSX

Issue/Introduction

• In NSX-T Federation, the Global Manager shows segments in UI with status: "Delete in Progress" when mousing over Segment name.
• On the Local Manager you may see similar log entries in the log file /var/log/syslog

2021-05-25T01:01:01.111Z nsxmgr NSX 6717 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" reqId="" subcomp="manager" username="nsx_policy"] Request to delete LogicalSwitch : LogicalSwitch/<LS_UUID>
2021-05-25T01:01:01.111Z nsxmgr NSX 6717 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" reqId="" subcomp="policy" username="admin"] Entity /global-infra/segments/<LS_UUID> does not exist

Environment

VMware NSX
VMware NSX-T Data Center

Cause

A segment deletion attempt from the Global Manager can become stuck in progress if a Local Manager fails to delete its Logical Switch for the corresponding Segment.

Resolution

1. Confirm in LM UI that there are no ports in use with the Logical Switch which was not deleted
 
2. To remove the Logical Switch, SSH to a Local Manager node as admin
 
3. Run get logical-switches on the Local Manager CLI and confirm the stale Logical Switch is listed, and note its UUID
 
4. Elevate to root shell with command st en (root password required)
 
5. Confirm the Logical Switch info can be polled with API:
curl -k -v -H "Content-Type:application/json" -u admin -X GET "https://{local_mgr_IP}/api/v1/logical-switches/(LS_UUID)"
 
6. Remove stale Logical Switch objects via API:
curl -k -v -H "Content-Type:application/json"  -u admin -X DELETE "https://{local_mgr_IP}/api/v1/logical-switches/{LS_UUID}?cascade=true&detach=true"
 
This will return '200' response code if deletion is successful
 
7. Once Logical Switch is removed from Local Manager in question, confirm Segment is automatically deleted in Local Manager and Global Manager UI
 

8. Perform a sync between the GM and each local manager, using the following API:
curl -k -v -H "Content-Type:application/json"  -u admin -X POST https://{local_mgr_IP}/policy/api/v1/infra/full-sync-action?action=request_notifications_full_sync