Stale Edge Undeploy in Progress

Products

• VMware NSX

Issue/Introduction

When trying to delete an edge from NSX. This will go into updeploy in progress and will not finish the task, also if you go into the Vcenter you will see the edge vm still power on. 

Environment

VMware NSX

Cause

• Initiating a delete operation on the NSX Edge node from the NSX UI causes the NSX manager to contact the edge directly for deletion.
• If the NSX manager fails to contact the Edge node, it tries to ask the vCenter for a delete operation (applies when the edge node was deployed via the NSX UI and not OVA).
• If both steps fail, the EDGE gets stuck in the "Delete in progress" state.

Resolution

Step 1 

• Make sure that the Edge has been deleted from Vcenter 
  • if this still exists on Vcenter 
    • power off the VM 
    • right-click on the VM or click on actions and click on delete from disk 

Note: if the edge VM does not exist, proceed with step 2 

Step 2 

After we have removed the edge VM from VCenter:

 

1. Attempt Edge Deletion:

   • Use the DELETE https://<manager-ip>/api/v1/transport-nodes/<tn-id> API.
     • This can be ran using curl or using tools like Postman please see NSX API guide.
   • For NSX-T version 3.2.1 and later, if the transport node is orphaned, the stale API clears table entries.
   • Note: This API won't work for versions prior to 3.2.1.

2. Monitor Deletion Progress:

   • The background retries (with exponential backoff) try to complete the deletion on the edge.
   • Users might observe changes in the /state API output.
   • The “state” field in /state API will transit from “pending” -> “in_progress” -> “failed” -> “orphaned”.
   • Check the status of deletion using GET https://<manager-ip>/api/v1/transport-nodes/<tn-id>/state API.

3. Identify Stuck Deletion:

   • If the deletion is stuck for more than 30 minutes, consider the scenario as stuck, If the Edge VM is still existing in NSX UI inventory after following the workaround, restart nsx-proxy service on the host which was hosting the Edge VM
     /etc/init.d/nsx-proxy status | restart

Additional Information

Troubleshooting Edge Deletion Failures in VMware NSX