NSX Host Disconnected: Mismatch Between Installed NSX VIBs and ESXi Version in UI

Products

• VMware NSX

Issue/Introduction

• ESXi host status in NSX Manager UI displays as "Disconnected," "Failed," or "Partially Configured."
• The reported "OS Type" or ESXi version for the affected host in the NSX UI is incorrect or older than expected.
• Problems with NSX-managed networking on the host, potentially impacting VM connectivity or services.
  
  

Environment

VMware NSX
VMware NSX-T Data center

Cause

The primary cause of the host being in a disconnected state with an incorrect OS type reported in the NSX UI is a fundamental mismatch: NSX VIBs designed for a newer ESXi version are installed on a host that NSX (or its underlying vCenter integration) perceives as an older version, or vice-versa, leading to a state of inconsistency.

Resolution

1. Remove the Host from the NSX Cluster/Prepare for Uninstallation:

   • Navigate to the NSX Manager UI.
   • Locate the disconnected host.
   • Initiate the process to remove the host from the NSX cluster (or unprepare it if it's still partially configured).

2. Recover Host Management Connectivity (if lost):

   • Crucial Step: If the host was using an NSX-managed distributed switch (N-VDS or VDS enhanced for NSX) for its management vmkernel (vmk0), removing it from the NSX cluster will likely cause it to lose management connectivity.
   • To regain access:
     • Access the ESXi host directly via the DCUI (Direct Console User Interface) or through iLO/DRAC/IPMI.
     • Create a new standard vSwitch on the host.
     • Migrate the management vmkernel (vmk0) from the NSX-managed switch to this newly created standard vSwitch.
     • Ensure the standard vSwitch has a physical uplink connected to the correct network for management.
     • Verify network connectivity to the host.
   • Reference KB: For detailed steps on recovering management access from a DVS:
     • Recovering management access to an ESXi host from a Distributed Switch (2032)

3. Migrate Other vmkernels (e.g., vMotion, vSAN):

   • Once management access is stable, migrate any other critical vmkernel adapters (e.g., vMotion, vSAN, Fault Tolerance) from the NSX-managed switch to either a standard vSwitch or another compatible DVS not managed by NSX, if they were also using the NSX switch. This ensures these services remain operational during the VIB cleanup.

4. Forcefully Delete NSX VIBs:

   • From the NSX Manager UI, after the host has been removed from the cluster or unprepared, you should now be able to initiate a forceful uninstallation of NSX VIBs from the host. This step aims to completely clean up any lingering NSX components that are causing the version mismatch.

5. Add the Host Back to the NSX Cluster:

   • Once NSX VIBs are confirmed to be removed and the host has stable management connectivity, add the ESXi host back into the appropriate NSX-T compute manager and cluster from the NSX Manager UI.
   • NSX will then proceed to install the correct and compatible VIBs for the ESXi version it recognizes.

6. Verify Connectivity and VIB Installation:

   • After the host is added back, monitor the NSX Manager UI to confirm that the host transitions to an "Up" or "Success" state.
   • Verify that the correct NSX VIBs are installed on the host and that it now communicates correctly with the NSX Controller.
   • Confirm that all NSX-managed networking on the host is functional.