NSX upgrade Fails or hang with connection UNKNOWN warning.
search cancel

NSX upgrade Fails or hang with connection UNKNOWN warning.

book

Article ID: 453057

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • During an upgrade to NSX 9.0.2, the process gets stuck and pauses at 98% in the "finalize upgrade" phase. You'll see the following warning pop up in the Upgrade Coordinator UI:

Connection between host and NSX Controller / NSX Manager is UNKNOWN

  • When reviewing the NSX inventory, multiple duplicate ESXi host entries with different IP addresses appear as Disconnected or UNKNOWN under Standalone Hosts.

Environment

  • VMware NSX 

 

Cause

The Upgrade Coordinator is getting blocked by stale and orphaned Transport Node records left behind in the NSX database. This usually happens when ESXi hosts are decommissioned, rebuilt, or given new IPs without being cleanly removed or unprepared from NSX first.

Because the database is still holding onto those old UUIDs, the Upgrade Coordinator keeps trying to reach these "ghost" hosts during its post-checks and execution. When it can't establish a heartbeat, the upgrade halts.

Resolution

To resolve this issue, you must manually remove the stale transport node entry from the NSX Manager database using the NSX API and reset the host upgrade plan.

Follow these steps:

  1. Force-Delete the Stale Transport Node

    Execute a force-delete of the stale transport node via the NSX Manager API. Run the following API call against the NSX Manager:

    DELETE https://<nsx-mgr-ip>/api/v1/transport-nodes/<transport-node-UUID>?force=true&unprepare_host=false

    Note: Ensure that this API call returns a 200 OK HTTP status code.


  2. Verify the Node Removal

    Confirm that the node has been completely removed from the database by running an API GET command against the Transport Node ID (TNID):

    GET https://<nsx-mgr-ip>/api/v1/transport-nodes/<transport-node-UUID>

    Note: This command should return a 404 Not Found error, confirming the node no longer exists in the database.

  3. Verify the hosts are no longer part of the upgrade groups and then you can resume or reinitiate the NSX upgrade process.
Powered by Wolken Software