NSX host transport node upgrade fails with error "Upgrade failed: Solution apply failed on host: <hostname>. The vSphere Lifecycle Manager image contains a new 'ESXi' version."

Products

• VMware NSX

Issue/Introduction

When performing an upgrade of host transport nodes, the process fails specifically for ESXi hosts while Edge nodes upgrade successfully. This issue occurs when using vSphere Lifecycle Manager (vLCM) and there is a discrepancy between the host's current software and the cluster's desired image state.

• Edge nodes are successfully upgraded.
• NSX host upgrade fails with the following error: Upgrade failed: Solution apply failed on host: [HOSTNAME]. The vSphere Lifecycle Manager image contains a new 'ESXi' version.
  
  
  
  The upgrade group shows following error (ESXi version can be different in the error message) -
  
  

Environment

VMware NSX

Cause

The issue happens because host is not compliant with the cluster image specified in vLCM. vSphere Lifecycle Manager (vLCM) image represents a desired software specification to be applied to all ESXi hosts in a vSphere cluster. If any host in the cluster is not running the same version then it is marked as non-compliant and needs to be remediated for NSX to complete the upgrade.

Host compliance can be verified by navigating to the cluster and checking status under 'Updates > Hosts > Image'.

Here we can see host version is different from image version so NSX upgrade is failing.

Resolution

The resolution requires the host to be remmediate first which would make the host version to match the image version and then attempt the upgrade from NSX Manager.

1. Navigate to the affected cluster in the vSphere Client.
2. Select Updates > Hosts > Image to verify host compliance status
3. Evacuate all virtual machines from the non-compliant host, or ensure it is in a maintenance window.
4. Select the host and click Actions > Remediate

5. Once remediation is complete and the host version matches the cluster image, return to the NSX Manager and retry the host upgrade.

Note: This will upgrade ESXi so evacuate the VMs from this host and perform this activity in a maintenance window.