Best Practice for Decommissioning an NSX-Prepared ESXi Host to Avoid Stale Entries

Products

• VMware NSX

Issue/Introduction

When an ESXi host that is prepared as an NSX transport node needs to be decommissioned (e.g., for hardware replacement or retirement), it is critical to follow a specific procedure to remove it from the environment. Failure to do so can result in stale or orphaned entries in the NSX Manager database. These stale entries can lead to operational issues, including misleading health statuses, inaccurate inventory, and critical failures during future NSX upgrades.

Environment

VMware NSX

Cause

Stale entries are created when a host is removed from the vCenter inventory or is powered down permanently without first being properly unprepared through the NSX Manager. The NSX Manager relies on communication with the host to clean up its own configuration database. If the host becomes unreachable before this process is completed, NSX Manager will retain the host's configuration, treating it as a "down" or "unreachable" node rather than a cleanly removed one.

 

Resolution

To cleanly decommission a host and avoid stale entries, follow this mandatory, sequential procedure. The guiding principle is to always initiate the uninstallation from the NSX Manager first.

Step 1: Place the ESXi Host into Maintenance Mode

• In the vSphere Client, locate the target ESXi host, right-click it, and select Maintenance Mode > Enter Maintenance Mode.
• Ensure that all virtual machines have been migrated off the host before proceeding. This is a critical prerequisite to prevent service disruption.

Step 2: Remove the NSX Configuration from the Host

• Log in to the NSX Manager UI and navigate to System > Fabric > Nodes > Host Transport Nodes.
• Find and select the host that is in maintenance mode.
• Click the "Remove NSX" button. This step will trigger NSX Manager to uninstall all NSX components and configurations from the host.

Step 3: Verify Successful Removal in NSX Manager

• Monitor the host's status on the "Host Transport Nodes" screen within NSX Manager.
• Wait until the "Configuration State" for the host changes to "Not Prepared" or "NSX Uninstalled". The NSX Version column should also become blank. Do not proceed until this step is confirmed.

Step 4: Decommission the Host from the vCenter Server

• Once NSX is fully uninstalled, return to the vSphere Client. Right-click the host (still in maintenance mode) and select Connection > Disconnect.
• After the host is disconnected, right-click it again and select "Remove from Inventory".

Step 5: Physical Decommissioning

• After the host has been cleanly removed from both NSX and vCenter, you may safely power down, un-rack, or repurpose the physical hardware.

Important Note: Under no circumstances should a host be removed from vCenter inventory or powered off before its NSX configuration has been successfully removed via the NSX Manager UI. Doing so will result in a stale configuration that may require manual database cleanup.

Additional Information

If you are encountering stale NSX host entries, likely due to incorrect decommissioning:

• Consult Troubleshooting KBs:
  • KB 372410: For preparation failures due to stale entries.
  • KB 387942: For identifying orphaned host entries.

• Consult General Documentation/KBs:

  • • NSX-T Data Center Installation Guide - Uninstall NSX from a Standalone ESXi Host
    • KB 375111: Troubleshooting NSX Uninstall Process
      
      
• Seek Support: If the issue remains unresolved or requires direct database manipulation, please open a Broadcom Technical support case. (Refer to Creating and managing Broadcom support cases for process details).