Sync/Update of HCL data fails with, "A general system error occurred:"

Products

VMware vSphere ESXi

Issue/Introduction

Sync of the HCL data fail with the following :

/var/log/lifecycle.log

ImageManagerCTL: 60496: YYYY-MM-DD HH:MM:SS, imagemanagerctl:976 INFO Calling with arguments: depots --validate --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/addon-main/vmw-depot-index.xml --depot https://hostupdate
.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/iovp-main/vmw-depot-index.xml --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/vmtools-main/vmw-de
pot-index.xml --depot https://vmwaredepot.dell.com/index.xml --depot /storage/updatemgr/patch-store/hostupdate/__hostupdate20-consolidated-index__.xml
ImageManagerCTL: 60496: YYYY-MM-DD HH:MM:SS Downloader:262 INFO Opening https://vmwaredepot.dell.com/index.xml for download
ImageManagerCTL: 60496: YYYY-MM-DD HH:MM:SS DepotCollection:271 INFO Could not download from depot at https://vmwaredepot.dell.com/index.xml, skipping (('https://vmwaredepot.dell.com/index.xml', '', "Unknown proxy protocol 'https'
"))
ImageManagerCTL: 60496: YYYY-MM-DD HH:MM:SS DepotMgr:137 ERROR Unable to connect to depot(s) /storage/updatemgr/patch-store/hostupdate/__hostupdate20-consolidated-index__.xml,https://vmwaredepot.dell.com/index.xml,https://hostupda
te.vmware.com/software/VUM/PRODUCTION/vmtools-main/vmw-depot-index.xml,https://hostupdate.vmware.com/software/VUM/PRODUCTION/iovp-main/vmw-depot-index.xml,https://hostupdate.vmware.com/software/VUM/PRODUCTION/addon-main/vmw-depot-inde
x.xml,https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml:
('https://vmwaredepot.dell.com/index.xml', '', 'Could not download from depot at https://vmwaredepot.dell.com/index.xml, skipping ((\'https://vmwaredepot.dell.com/index.xml\', \'\', "Unknown proxy protocol \'https\'"))')
Traceback (most recent call last):
File "/usr/lib/vmware-imagebuilder/site-packages/vmware/esximage/Downloader.py", line 264, in _openfromurl
return self._retry(self._open_for_download, self.options['retry'])
File "/usr/lib/vmware-imagebuilder/site-packages/vmware/esximage/Downloader.py", line 193, in _retry
return func()
File "/usr/lib/vmware-imagebuilder/site-packages/vmware/esximage/Downloader.py", line 201, in _open_for_download
robj = self._urlopen(self.url)
File "/usr/lib/vmware-imagebuilder/site-packages/vmware/esximage/Downloader.py", line 185, in _urlopen
return opener.open(url, timeout=self.options['timeout'])
File "/usr/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/vmware-imagebuilder/site-packages/vmware/esximage/Utils/Proxy.py", line 191, in _https_open
h = HTTPSProxyConnection(host, proxy=self.proxies['https'])
File "/usr/lib/vmware-imagebuilder/site-packages/vmware/esximage/Utils/Proxy.py", line 118, in __init__
raise ValueError("Unknown proxy protocol '%s'" % parsedResult.scheme)
ValueError: Unknown proxy protocol 'https'

/var/log/vmware/vmware-updatemgr/vum-server/hcl_python_lib.log:

YYYY-MM-DD HH:MM:SS ERROR compatibility.vvs.rest.vvs_download Download client creation failed!
Traceback (most recent call last):
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_download.py", line 83, in vvsDownloadClientFactory
return VvsDownloadClient(configuration=configuration)
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_download.py", line 30, in __init__
super(VvsDownloadClient, self).__init__(configuration)
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_client.py", line 53, in __init__
key_file=configuration.keyKile,
AttributeError: 'Configuration' object has no attribute 'keyKile'
YYYY-MM-DD HH:MM:SS ERROR __main__ Internal error occurred
Traceback (most recent call last):
File "/usr/lib/vmware-updatemgr/python/hcl/hcl_datastore.py", line 277, in main
result = callback(**callbackArgs)
File "/usr/lib/vmware-updatemgr/python/hcl/hcl_datastore.py", line 192, in updateOnline
vvsDownloadClientFactory().downloadData(fileName)
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_download.py", line 86, in vvsDownloadClientFactory
raise e
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_download.py", line 83, in vvsDownloadClientFactory
return VvsDownloadClient(configuration=configuration)
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_download.py", line 30, in __init__
super(VvsDownloadClient, self).__init__(configuration)
File "/usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_client.py", line 53, in __init__
key_file=configuration.keyKile,
AttributeError: 'Configuration' object has no attribute 'keyKile'

/var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server-9.log:

YYYY-MM-DD HH:MM:SS info vmware-vum-server[107376] [Originator@6876 sub=RunCommand] Run command /usr/bin/bash -c "/usr/lib/vmware-updatemgr/image-manager/image-manager-ctl depots --validate --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/addon-main/vmw-depot-index.xml --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/iovp-main/vmw-depot-index.xml --depot https://hostupdate.vmware.com/software/VUM/PRODUCTION/vmtools-main/vmw-depot-index.xml --depot https://vmwaredepot.dell.com/index.xml --depot /storage/updatemgr/patch-store/hostupdate/__hostupdate20-consolidated-index__.xml"
-->
YYYY-MM-DD HH:MM:SS error vmware-vum-server[107376] [Originator@6876 sub=DepotContentManager] Depot Validation: Problems found while validating depot content: {
--> "info": [],
--> "warnings": [],
--> "errors": [
--> {
--> "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.MetadataDownloadError",
--> "message": {
--> "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.MetadataDownloadError",
--> "default_message": "An error occurred while downloading depot metadata from {1}.",
--> "args": [
--> "https://vmwaredepot.dell.com/index.xml"
--> ]
--> },
--> "resolution": null,
--> "time": "YYYY-MM-DD HH:MM:SS"
--> }
--> ]
--> }
YYYY-MM-DD HH:MM:SS error vmware-vum-server[107376] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.SyncDepotsTask] Failed to sync depots. Merged depot content is invalid: Error:
--> com.vmware.vapi.std.errors.error
--> Messages:
--> com.vmware.vcIntegrity.lifecycle.EsxImage.MetadataDownloadError

Cause

The SyncHCL task triggers the execution of the vvs_client.py script, as shown in the trace above.

This script contains two sections of code: one for handling HTTP connections through a proxy, and another for HTTPS connections through a proxy. The issue occurs in the HTTPS section due to a typo, the parameter is incorrectly written as "KeyKile" instead of "KeyFile". As a result, the error occurs only when establishing HTTPS connections.

Resolution

This issue is resolved in VMware vCenter version 7.0U1.

To workaround the issue :

Apply the following temporary fix. Please make sure to take a backup of the file before making any changes.

Open the file: /usr/lib/vmware-updatemgr/python/hcl/compatibility/vvs/rest/vvs_client.py

Navigate to line 53. You will see a code block similar to the following:

46 if configuration.proxy:
47 self.http = urllib3.ProxyManager(
48 num_pools=1,
49 maxsize=maxsize,
50 cert_reqs=certReqs,
51 ca_certs=configuration.SSLCACert,
52 cert_file=configuration.certFile,
53 key_file=configuration.keyKile,
54 proxy_url=configuration.proxy
55
56 )

Note: The problem is on line 53, where key_file=configuration.keyKile contains a typo.

Replace the incorrect line:
```
key_file=configuration.keyKile,

with

key_file=configuration.keyFile,
```
Note: Python is indentation-sensitive. Ensure the spaces at the beginning of the line remain consistent.
Run "Sync HCL" again.