NSX Upgrade Pre-Check Fails Due to Stale Configuration on Decommissioned ESXi Hosts

Products

• VMware NSX

Issue/Introduction

During an NSX upgrade, the upgrade pre-check task fails, preventing the upgrade process from proceeding. The pre-check logs or NSX UI display errors related to ESXi hosts, such as "Manager connectivity check" failures, "unmanaged host" alarms, or "MPA connectivity down" alerts.

Symptoms

• NSX Upgrade pre-check task returns FAILURE for specific ESXi hosts.
• Connectivity checks fail on hosts that are no longer actively part of the cluster or have been decommissioned.
• The NSX Fabric interface may still show decommissioned hosts with stale state information.

Environment

VMware NSX

Cause

The upgrade pre-check process attempts to validate all transport nodes (TNs) associated with the NSX fabric. If an ESXi host has been decommissioned or removed from the vCenter environment without first being explicitly removed from the NSX Fabric, the NSX Manager retains stale configuration for that node. The pre-check validation fails because the Manager cannot establish connectivity to the removed or unreachable host.

Resolution

To resolve these pre-check errors and proceed with the upgrade, remove the stale NSX configuration for the decommissioned hosts:

1. Identify the Stale Host(s):

   • Review the NSX upgrade pre-check logs to identify the specific host UUIDs or FQDNs generating the FAILURE messages.
   • Cross-reference these with your current inventory to confirm they are indeed decommissioned or no longer in use.

2. Remove from NSX Fabric:

   • Log in to the NSX Manager UI.
   • Navigate to System > Fabric > Nodes.
   • Locate the decommissioned host(s) identified in Step 1.
   • Ensure the host is selected and proceed to remove it from the NSX Fabric configuration. (Note: Depending on your NSX version and configuration, this may be found under the "Standalone" or "Host Transport Nodes" tabs).

3. Verify and Retry:

   • Once the host has been successfully removed from the NSX Fabric, re-run the NSX upgrade pre-check task.
   • The errors should clear, and the upgrade pre-check should pass, allowing you to proceed with the upgrade.

Additional Information

• Best Practice: Always perform a clean removal of NSX configuration (via the NSX UI) before decommissioning or unregistering an ESXi host from vCenter.
• Related Errors: If errors persist after removing the host, ensure there are no stale transport node objects in the API that require manual deletion.