Unable to enable vSphere HA on Image-based cluster after vCenter upgrade, crashing update Manager

Products

• VMware vCenter Server

Issue/Introduction

• vCenter has been upgraded from 8.0u3d(build 24322831) to 8.0u3e(build 24674346)
• Unable to enable vSphere HA on Image-based cluster post upgrade with either of the below errors: 
  
  Cannot complete the configuration of the vSphere HA agent on the host. "Setting desired image spec for cluster failed." 
Cannot complete the configuration of the vSphere HA agent on the host. "Applying HA VIBs on the cluster encountered a failure."
  
  
• The cluster is Image-based and there's vsphere-fdm of version 8.0.3-24322831 added as a component to the image
• When trying to Enable HA in the cluster and this fail, this cause update manager services to stop creating dump file at on /var/core.
• Below message is observed in /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log of vCenter which also indicates there's a vsphere-fdm of legacy version 8.0.3-24322831 present in the image as a component while adding the newer version of 8.0.3-24674346 as a solution:
  
  vmware-vum-server[159915] [Originator@6876 sub=ServiceProvider] [EmbeddedPyServiceProvider 2263] The software spec string: {
-->     "add_on": null,
-->     "alternative_images": null,
-->     "base_image": {
-->         "version": "<ESXi base version>"
-->     },
-->     "components": {
-->         "vsphere-fdm": "8.0.3-24322831"
-->     },
-->     "hardware_support": null,
-->     "removed_components": null,
-->     "solutions": {
-->         "com.vmware.vsphere-ha": {
-->             "components": [
-->                 {
-->                     "component": "vsphere-fdm"
-->                 }
-->             ],
-->             "version": "8.0.3-24674346"
-->         }
-->     }
--> }
  
  Compare the log version above with the version listed in the host and vCenter, as shown in the example below.
  Note: The version number may vary depending on the installed vSphere version
  
  Host: 
  # esxcli software vib list | grep -i fdm
  vmware-fdm                     8.0.3-24674346
  
  vCenter
  # ls -lh /etc/vmware-vpx/docRoot/vSphere-HA-depot/vib20/vmware-fdm
  
  VMware_bootbank_vmware-fdm_8.0.3-1.24674346.vib
  VMware_bootbank_vmware-fdm_8.0.3-24674346.vib

Environment

vCenter Server 8.0

Cause

There's a legacy version of vsphere-fdm present as component in lifecycle manager image which caused issue when adding vsphere-fdm of newer version as a solution. 

Resolution

The legacy version of vsphere-fdm component needs to be removed from the image to solve the issue. 

If the component cannot be removed from vCenter UI, an alternative way to get the component removed is:

1. Export the image as JSON file
2. Edit the JSON file and remove the vsphere-fdm component
3. Import the updated JSON file to the image
4. Validate the image and save

vSphere HA can be successfully enabled post removing legacy vsphere-fdm component. 

Additional Information

 

To verify the old FDM from the images on the cluster follow below steps : 

• In the vSphere Client, go to Hosts and Clusters and select cluster.

• Click the Updates tab, then select Image.

• In the Components list, locate the vsphere-fdm entry--> it will be showing the old build.