"Solution specification in the image are incompatible with hosts" error when performing an NSX upgrade of ESX Hosts managed by vLCM

Products

• VMware NSX

Issue/Introduction

• When performing an NSX upgrade a failure occurs stating
  
  Upgrade failed: Solution specification in the image are incompatible with hosts '<names>'

  
  
  
• In vCenter, the vLCM remediation task fails with an error:
  
  A general system error occurred: Solution apply failed because the vSphere Lifecycle image contains either a new ESXi version or a new addons version or new components. Please proceed to the vSphere Client Lifecycle Manager to update ESXi or addons or components along with the solution 'com.vmware.nsxt'
  
  
• On the NSX manager UI, as error similar to the following would be seen :
• 

Environment

VMware NSX 4.x
VMware NSX-T Datacenter

Cause

While running an upgrade through the NSX Manager UI, during the ESXi transport node upgrade stage, the upgrade tasks will start and proceed up until the point of the failure occurring on vLCM. The tasks will fail on NSX Manager UI with the described error message soon after remediation fails on vLCM. 

For vLCM managed clusters, NSX Manager sends the new NSX host components to vLCM. vLCM then installs the NSX VIBs through image remediation. A conflict between the version of components on the current image and the updated version will cause the remediation task to fail. This is to protect the system from a version downgrade.

Resolution

1. Refer to Option 1 in the resolution section of article, vSphere Lifecycle Manager Image Compliance check fails with Error: Host not compatible with the image, or Downgrades of manually added Components with instructions for editing VLCM settings for the hosts in vCenter to "select a version of the Vendor Addon that contains higher versions of the components".
2. With a version selected that no longer causes a component downgrade problem for VLCM, a remediation can be performed in vLCM.
3. Since the NSX VIBs would already have been added to vLCM, running the remediation through vLCM at this point will also update the NSX VIBs on the hosts.
4. Once complete, the ESXi transport node upgrade can be run again through the NSX Manager which should detect the current version and mark the ESXi hosts' upgrade as complete.