ESXI upgrade through Lifecycle Manager fails at Pre-check stage with error "NSX is not present is cluster desired state".

Products

• VMware vCenter Server

Issue/Introduction

The Pre-check stage of ESXi upgrade fails with error "NSX is not present is cluster desired state"

 

vmware-vum-server.log highlights the failure to access the depot content - /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log

YYYY-MM-DD error vmware-vum-server[1355373] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.CreateOfflineDepotTask] [CreateOfflineDepotTask 313] Failed to validate depot content

imageservice.log highlights the failure that the old NSX offline bundle in the depot is not cleaned up  - /var/log/vmware/vmware-updatemgr/vum-server/imageservice.log 

YYYY-MM-DD ERROR imageService[139798242911808] [DepotMgr 51] Unable to connect to depot(s) /storage/updatemgr/patch-store-temp/vum3719744378656892809.zip: VIBs VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 and VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 have unequal values of the 'payloads' attribute: '[nsx-cfgagent-es: 1654.213 KB]' != '[nsx-cfgagent-es: 2071.697 KB]'
YYYY-MM-DD ERROR imageService[139798242911808] [DepotMgr 51] Unable to connect to depot(s) /storage/updatemgr/patch-store-temp/vum3719744378656892809.zip: VIBs VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 and VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 have unequal values of the 'payloads' attribute: '[nsx-cfgagent-es: 1654.213 KB]' != '[nsx-cfgagent-es: 2071.697 KB]'
YYYY-MM-DD ERROR imageService[139798242911808] [DepotMgr 51] Unable to connect to depot(s) /storage/updatemgr/patch-store-temp/vum3719744378656892809.zip: VIBs VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 and VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 have unequal values of the 'payloads' attribute: '[nsx-cfgagent-es: 1654.213 KB]' != '[nsx-cfgagent-es: 2071.697 KB]'
YYYY-MM-DD ERROR imageService[139798242911808] [imageService 628] Failed to validate depot: VIBs VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 and VMware_bootbank_nsx-cfgagent-esxio_4.2.0.2.0-8.0.24278657 have unequal values of the 'payloads' attribute: '[nsx-cfgagent-es: 1654.213 KB]' != '[nsx-cfgagent-es: 2071.697 KB]

 

Environment

VMware vCenter Server 

Cause

The issue is encountered due to conflicting depots, one from the NSX uploaded depot or another offline depot that was manually uploaded. The offline/manually uploaded depot is causing the new depot upload to fail.

Resolution

1. Verify two different NSX-LCP bundles/depots are available on the vCenter - 

• SSH to vCenter with root credentials.
• Run - dcli +i
• Run the command to validate the list of depots - dcli> com vmware esx settings depots offline list

2. Reset the VMware Update Manager Database following the steps in the KB, along with the additional steps for NSX cluster  - Resetting the VMware Update Manager Database

3. Re-trigger the Prepare Cluster task from the NSX manager to push the NSX depot on the vCenter.