NSX vibs installation on hosts failed with error Failed to install software on host. Unable to connect the host https://<IPaddress>/sdk after re-adding the hosts to the cluster

Products

• VMware NSX
• VMware vSphere ESX 7.x
• VMware Telco Cloud Platform

Issue/Introduction

ESXi hosts which were prepared for NSX were taken out of cluster for maintenance activities. While re-adding the ESXi hosts to the same cluster, NSX vibs installation on the hosts failed with the failure as shown below 

NSX installation starts but fails almost immediately with the error "Failed to install software on host. Unable to connect the host https://<IPaddress>/sdk . (here the IP address is incorrect management IP address of ESXi)

Environment

VMware ESXi, 7.0.3, 24585291

NSXT Version 3.2.2.0.0.20737185

TCP 2.2

 

Cause

The NSX vibs installation failed on ESXi host since the NSX manager was attempting to connect via incorrect IP address of the ESXi host.

Resolution

Perform below actions to recover the ESXi hosts and to re-install NSX vibs 

1. Keep the ESXi host in maintenance mode 
2. Check if any partial vibs are installed or any NSX configuration still remains on the ESX host. The presence of NSX vibs can be validated using command esxcli software vib list | grep -i nsx 
3. If NSX vibs are present, force delete the vibs using command nsxcli -c del nsx 
4. Ensure NSX vibs are removed completely and reboot the ESXi host.
5. After the ESXi host comes online and operational, confirm no NSX vibs still remain on the ESXi host before adding the host to NSX prepared cluster 
6. After adding the host to cluster, ensure the NSX vibs are getting installed by checking /var/run/log/esxupdate.log 

If the issue still persists, refer KB 206567 to contact Broadcom support

Additional Information

To uninstall NSX from ESXi refer Uninstall NSXT 3.2