ESXi host showing "Install Failed"

Products

• VMware NSX

Issue/Introduction

• When hovering of "Install Failed" in NSX Manager UI under "system -> Fabric -> Hosts":
  

  Failed to install software on host. Solution apply failed because the vSphere Lifecycle image contains either a new ESXi version or a new addons version or new components. Please proceed to the vSphere Client Lifecycle Manager to update ESXi or addons or components along with the solution 'com.vmware.nsxt' Solution apply failed on host: '<HOST-NAME>'. The vSphere Lifecycle Manager image contains a new 'Cisco-UCS-Addon-ESXi' addon version '4.3.6-a'. Please proceed to the vSphere Client Lifecycle Manager to update the addon and the solution 'com.vmware.nsxt'Solution apply failed on host: '<HOST-NAME>'

  
  
  
• In vCenter, under the Cluster --> Upgrade , the host will show as "not in compliance with cluster image".

Environment

VMware NSX

Cause

• When adding ESX hosts in a vLCM managed cluster in vCenter, the host must be in compliance with the cluster image. 

Resolution

• In vCenter, place the ESXi host into Maintenance mode. 
• Navigate to the Cluster level where the host exists. 
• Go to the "Upgrade" tab and remediate problematic host. 

Once the host has been remediated, navigate back to NSX UI where the error is presented. Click on the error and select "resolve". If NSX does not begin the install process, follow the steps below. (This may take up to 5 mins to display in the UI).

• While the host is still in MM, move the host out of the cluster into the Datacenter level. 
  • This should begin the process of uninstalling vibs from the host. 
    • Be sure there are no interfaces using the NSX switch and its components. 
    • In certain situations, the NSX-KCP module will block the NSX uninstall process. Follow the steps in KB:312644 to identify and unblock the NSX uninstall process.
    • Once these steps are complete, navigate back to NSX UI -->"hosts" tab and click on "standalone".
    • Select the host and then "remove NSX", this should restart the NSX uninstall process. 
      • If this fails, select "remove NSX" again but this time select "Force Delete" option. 
      • This WILL remove the host from NSX UI.
      • At this point you will need to SSH to the ESXi host in MM and run the command: "nsxcli -C del nsx"
        • You can monitor the removal of NSX vibs using the command : "tail -f /var/run/log/esxupdate.log"
        • Running the command "esxcli software vib list | grep -i nsx" will display any NSX vibs still present on the host. 
  • Now that the NSX vibs have been removed and no longer seen via the "esxcli software vib list" command , you must proceed with host reboot. 
  • After host has booted you may re-add to the cluster. NSX install will begin automatically.  
  • Monitor the NSX Install from the NSX UI.
  • Once this completes you can remove the host from MM and verify connectivity. 

Additional Information

• NSX VIBs are not removed from an ESXi host