Creating a VI Workload domain or adding a cluster fails at 'Failed to upload Image Zip to vcenter'

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• The "UploadZipToVcenterAction" sub-task fails during add cluster or create VI Workload Domain workflow with DownloadFail error:

Failed to upload Image Zip [http://sddc-manager.vrack.vsphere.local:7101/vmware/vcf/personalities/ce6b4e40-706d-4f58-967d-786e08b5734a/Content/OFFLINE_BUNDLE_1500756841.zip ] to vcenter.

• Visible error in the vCenter Web Client:

"An error occurred while downloading depot metadata from <image json file>"

• Visible in vCenter logs:

2020-03-30T23:54:15.721Z error vmware-vum-server[40115] [Originator@6876 sub=DepotContentManager]  Depot Validation: Problems found while validating depot content: {
    "info": [],
    "warnings": [],
    "errors": [
       {
          "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.MetadataDownloadError",
          "message": {
             "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.MetadataDownloadError",
             "default_message": "An error occurred while downloading depot metadata from {1}.",
             "args": [
                "zip:/storage/updatemgr/patch-store-temp/01bc-1d6d-3bf4-2608.zip?index.xml"
             ]
          },
          "resolution": null,
          "time": "2020-03-30T23:54:15.672Z"
       }
    ]
 }

Environment

VMware Cloud Foundation 4.x

Cause

The personality json file was not uploaded successfully to LCM.

Resolution

1. SSH to the VMware SDDC Manager VM and change to the root user:

su -

2. Navigate to the directory /nfs/vmware/vcf/nfs-mount/personalities/<Personality UUID>  where the PersonalityName is the one associated with the the domain/cluster being used.
3. Using Step 1 from this documentation, download the LCM personality zip from the vCenter Server.
4. Using a file transfer utility, upload the file to /nfs/vmware/vcf/nfs-mount/personalities/<PersonalityName> to replace the zip file with the correct personality zip. 
5. Make sure to keep the Offline bundle zip name the same.
6. Restart the create VI/ Add Cluster operation from the SDDC Manager UI.

Additional Information

For determining the personality name, steps in the following document can be used to export from the vCenter Server, https://docs.vmware.com/en/VMware-Cloud-Foundation/4.0/com.vmware.vcf.admin.doc_40/GUID-3688CAC2-06E0-4422-8CFA-0332F373FD7E.html