ESXi Host seeding is failing on "extract on custom content autobootstrapped host"

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

ESXi Host seeding fails under the below conditions:

• Create vLCM with Base Image + vendor addOn.
• Create Autodeploy rule having vLCM cluster as target.
• Autobootstrap the host to vLCM cluster
• Now seed from the autobootstrapped host (Here seeding will fail at extract)

On the seeding ESXi Host /var/run/log/lifecycle.log you may see similar entries:

2022-01-06T12:31:21Z lifecycle: 1060656: HostSeeding:956 ERROR Extract depot failed: ('MEL_bootbank_nmlx4-en_3.19.70.1-1OEM.670.0.0.8169922', 'Cannot find or stage payload nmlx4-en type vgz of VIB MEL_bootbank_nmlx4-en_3.19.70.1-1OEM.670.0.0.81
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:373 ERROR Extract depot failed.
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:152 ERROR [VibRecreateError] ('MEL_bootbank_nmlx4-en_3.19.70.1-1OEM.670.0.0.8169922', 'Cannot find or stage payload nmlx4-en type vgz of VIB MEL_bootbank_nmlx4-en_3.19.70.1-1OEM.670.0.0.8
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR Traceback (most recent call last):
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/Depot.py", line 928, in GenerateVib
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR stagePath = getBootBankPayloadPath(vibobj, name, hostImage,
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/ImageManager/HostSeeding.py", line 211, in getBootBankPayloadPath
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR _checkPayloadChecksum(fObj, pObj)
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/ImageManager/HostSeeding.py", line 115, in _checkPayloadChecksum
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR raise RuntimeError('Calculated %s checksum of payload %s does not '
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR RuntimeError: Calculated sha256 checksum of payload nmlx4-en does not match VIB metadata
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR During handling of the above exception, another exception occurred:
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR Traceback (most recent call last):
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/usr/lib/vmware/lifecycle/bin/imagemanagerctl.py", line 371, in depots
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR HostSeeding.InstalledImageInfo().extractDepot(task)
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/ImageManager/HostSeeding.py", line 942, in extractDepot
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR Depot.DepotFromImageProfile(newProfile, depotDir,
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/Depot.py", line 1032, in DepotFromImageProfile
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR vibdownloadfn(localfn, allRelatedVibs[vibid],
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR File "/lib64/python3.8/site-packages/vmware/esximage/Depot.py", line 938, in GenerateVib
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR raise Errors.VibRecreateError(vibobj.id, msg)
2022-01-06T12:31:21Z lifecycle: 1060656: imagemanagerctl:157 ERROR vmware.esximage.Errors.VibRecreateError: ('MEL_bootbank_nmlx4-en_3.19.70.1-1OEM.670.0.0.8169922', 'Cannot find or stage payload nmlx4-en type vgz of VIB MEL_bootbank_nmlx4-en_3.

Environment

VMware vSphere ESXi 7.0.3

Resolution

To resolve this issue, you may need to seed from ESXi Host which is not autobootstrapped.

Workaround:
Workaround:
1. Add 7.0 U3C host to vLCM cluster where cluster image is 7.0 U3C + vendor addOn.
2. Remediate the 7.0 U3C ESXi host to install vendor addOn.
3. Seed from the ESXi host