vSphere HA configuration fails with "error: A general system error occurred: Solution being applied does not exist in the desired image" or "Image is not valid"

Products

• VMware vCenter Server

Issue/Introduction

• vSphere HA configuration fails with message "Image is not valid" or "Solution being applied does not exist in the desired image" on an imaged-based cluster.
• Under /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log, below log snippets are found:
  
  YYYY-MM-DD-THH:MM:SS warning vmware-vum-server[#####] [Originator@6876 sub=TaskStatsCollector] [taskStatsCollector ###] Task type or creation time not present
YYYY-MM-DD-THH:MM:SS info vmware-vum-server[#####] [Originator@6876 sub=PM.AsyncTask.SetSolutionTask{520}] [vciTaskBase ####] SerializeToVimFault fault:
--> (vmodl.fault.SystemError) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = (vmodl.LocalizableMessage) [
-->       (vmodl.LocalizableMessage) {
-->          key = "com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound",
-->          arg = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "1",
-->                value = "com.vmware.vsphere-ha"
-->             },
-->             (vmodl.KeyAnyValue) {
-->                key = "2",
-->                value = "#########"
-->             }
-->          ],
-->          message = <unset>
-->       },
-->       (vmodl.LocalizableMessage) {
-->          key = "com.vmware.vcIntegrity.lifecycle.softwarespec.InvalidImage",
-->          arg = <unset>,
-->          message = <unset>
-->       }
-->    ],
-->    reason = "vLCM Task failed, see Error Stack for details."
-->    msg = "{
-->     "data": {
-->         "notifications": {
-->             "errors": [
-->                 {
-->                     "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound",
-->                     "message": {
-->                         "args": [
-->                             "com.vmware.vsphere-ha",
-->                             "#########"
-->                         ],
-->                         "default_message": "Software Solution com.vmware.vsphere-ha with version ######### cannot be found in depot.",
-->                         "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound",



YYYY-MM-DDTHH-MM-SS error vmware-vum-server[#####] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.DeleteSolutionTask] [Task, 524] Task:com.vmware.vcIntegrity.lifecycle.DeleteSolutionTask ID:########################. Task Failed. Error: Error:
-->    com.vmware.vapi.std.errors.not_found
--> Messages:
-->    com.vmware.vcIntegrity.lifecycle.TaskError.SolutionNotFound<Solution being applied does not exist in the desired image.>
-->
YYYY-MM-DDTHH-MM-SS warning vmware-vum-server[#####] [Originator@6876 sub=TaskStatsCollector] [taskStatsCollector ###] Task type or creation time not present
YYYY-MM-DDTHH-MM-SS info vmware-vum-server[#####] [Originator@6876 sub=PM.AsyncTask.DeleteSolutionTask{521}] [vciTaskBase ####] SerializeToVimFault fault:
--> (vmodl.fault.SystemError) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = (vmodl.LocalizableMessage) [
-->       (vmodl.LocalizableMessage) {
-->          key = "com.vmware.vcIntegrity.lifecycle.TaskError.SolutionNotFound",
-->          arg = <unset>,
-->          message = <unset>
-->       }
-->    ],
-->       }
-->    ]
-->    msg = "{
-->     "data": null,
-->     "error_type": "NOT_FOUND",
-->     "messages": [
-->         {
-->             "args": [],
-->             "default_message": "Solution being applied does not exist in the desired image.",
-->             "id": "com.vmware.vcIntegrity.lifecycle.TaskError.SolutionNotFound"
-->         }
-->     ]
--> }"
--> }
  
  
  
  
• Under /var/log/vmware/vmware-updatemgr/vum-server/imageservice.log, below log snippets are found:
  
  YYYY-MM-DDTHH-MM-SS INFO imageService [SoftwareSpecMgr ####] Image validation result: {'info': [], 'warnings': [], 'errors': [{'id': 'com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound', 'message': {'id': 'com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound', 'default_message': 'Software Solution com.vmware.vsphere-ha with version ######### cannot be found in depot.', 'args': ['com.vmware.vsphere-ha', '#########']}, 'resolution': None, 'time': 'YYYY-MM-DDTHH-MM-SS'}]}
YYYY-MM-DDTHH-MM-SS INFO imageService [SoftwareSpecMgr ####] Image validation result: {'info': [], 'warnings': [], 'errors': [{'id': 'com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound', 'message': {'id': 'com.vmware.vcIntegrity.lifecycle.EsxImage.SolutionNotFound', 'default_message': 'Software Solution com.vmware.vsphere-ha with version ######### cannot be found in depot.', 'args': ['com.vmware.vsphere-ha', '#########']}, 'resolution': None, 'time': 'YYYY-MM-DDTHH-MM-SS'}]}
  
  

Cause

The image being used at the Cluster level is either Invalid or has missing components.

Resolution

• Disable and re-enable the vSphere HA at the Cluster level.

Note:

• If the above option does not fix, then follow below steps.
• Ensure there is valid backup/offline snapshot(s) of the vCenter server(s) prior to implementing this resolution. Refer to VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice
• This option will reset the VUM Database and remove the image configured for the ESXi Hosts on all the Cluster(s).
• DO NOT perform these steps, if there is NSX in the environment.

1. Disable the vSphere HA.
2. SSH to the vCenter via root credentials.
3. And reset the Update Manager Database. Refer to KB Resetting the VMware Update Manager Database for more information.
4. Disable the default repositories and configure new repositories using the URLs that include the download token, as described in Broadcom KB 390121. To generate the download tokens, refer to the instructions for updating VCF authenticated download configurations.
5. Synchronize the vSphere Lifecycle Manager Depot following the steps mentioned in the document Working With the vSphere Lifecycle Manager Depot
6. Validate and save the cluster Image following the document Working with Images
7. Re-enable vSphere HA.

Additional Information

In NSX environment, You may look for an alternate solution as mentioned in KB Error: "Software Solution #### with version # cannot be found in depot" when reconfiguring for vSphere HA or contact VMware by Broadcom Support.