hbr VIB Installation Fails with Error 'An Error Occurred During Host Configuration'

Products

• VMware Live Recovery

Issue/Introduction

Symptoms: 

Continuous error tasks in VCenter " An error occurred during host configuration for hbr vib installation" 

Below Taks event in host

Task Name - Install
Status - An error occurred during host configuration.
Initiator - com.vmware.vr-sa-########-####-####-#####-#######@vsphere.local

Noticed /var/log/esxupdate.log below events

2024-11-13T07:00:24Z esxupdate: 5076018: esxupdate: INFO: nosigcheck = None
2024-11-13T07:00:25Z esxupdate: 5076018: HostImage: INFO: Installers initiated are {'live': <vmware.esximage.Installer.LiveImageInstaller.LiveImageInstaller object at 0x8ad68f1610>, 'boot': <vmware.esximage.Installer.BootBankInstaller.BootBankInstaller object at 0x8ad6b092b0>, 'locker': <vmware.esximage.Installer.LockerInstaller.LockerInstaller object at 0x8ad68f18e0>}
2024-11-13T07:00:25Z esxupdate: 5076018: imageprofile: INFO: Adding VIB VMware_locker_tools-light_12.2.6.22229486-22348808 to ImageProfile (Updated) ################
2024-11-13T07:00:25Z esxupdate: 5076018: imageprofile: DEBUG: Adding Component VMware-VM-Tools_12.2.6.22229486-22348808 to ImageProfile (Updated) ################
2024-11-13T07:00:25Z esxupdate: 5076018: downloader: INFO: Opening https://VR :443/vib/vmware-hbr-agent.vib for download
2024-11-13T07:00:58Z esxupdate: 5076018: downloader: WARNING: Download failed: <urlopen error _ssl.c:1114: The handshake operation timed out>, 4 retry left...
2024-11-13T07:01:31Z esxupdate: 5076018: downloader: WARNING: Download failed: <urlopen error _ssl.c:1114: The handshake operation timed out>, 3 retry left...
2024-11-13T07:02:04Z esxupdate: 5076018: downloader: WARNING: Download failed: <urlopen error _ssl.c:1114: The handshake operation timed out>, 2 retry left...
2024-11-13T07:02:37Z esxupdate: 5076018: downloader: WARNING: Download failed: <urlopen error _ssl.c:1114: The handshake operation timed out>, 1 retry left...
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: An esxupdate error exception was caught:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: Traceback (most recent call last):
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/urllib/request.py", line 1354, in do_open
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/http/client.py", line 1259, in request
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/http/client.py", line 1305, in _send_request
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/http/client.py", line 1254, in endheaders
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/http/client.py", line 1014, in _send_output
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/http/client.py", line 954, in send
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/http/client.py", line 1428, in connect
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/ssl.py", line 500, in wrap_socket
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/ssl.py", line 1040, in _create
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/ssl.py", line 1309, in do_handshake
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: socket.timeout: _ssl.c:1114: The handshake operation timed out
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: During handling of the above exception, another exception occurred:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: Traceback (most recent call last):
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 374, in _openfromurl
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     return self._retry(self._open_for_download, self.options['retry'])
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downnnnnnloader.py", line 224, in _retry
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     _checkRetry(retry, retries, e)
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 208, in _checkRetry
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     raise exception
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 214, in _retry
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     return func()
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 264, in _open_for_download
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     robj = self._urlopen(self.url)
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 198, in _urlopen
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     return opener.open(url, timeout=self.options['timeout'])
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/urllib/request.py", line 525, in open
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/urllib/request.py", line 542, in _open
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/urllib/request.py", line 502, in _call_chain
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/urllib/request.py", line 1397, in https_open
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/urllib/request.py", line 1357, in do_open
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: urllib.error.URLError: <urlopen error _ssl.c:1114: The handshake operation timed out> 
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: During handling of the above exception, another exception occurred:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: Traceback (most recent call last):
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 881, in _getVibsFromUrls
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     with d.Open() as rObj:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 473, in Open
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     return self._openfromurl()
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Downloader.py", line 376, in _openfromurl
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     raise DownloaderError(self.url, self.local, str(e))
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: vmware.esximage.Downloader.DownloaderError: ('https://VR:443/vib/vmware-hbr-agent.vib', '', '<urlopen error _ssl.c:1114: The handshake operation timed out>')
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: During handling of the above exception, another exception occurred:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: Traceback (most recent call last):
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/usr/sbin/esxupdate", line 222, in main
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     cmd.Run()
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esx5update/Cmdline.py", line 148, in Run
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 2215, in InitVibInstall
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     Transaction.GetVibsAndMetaFromSources(vibUrls, metaUrls, vibSpecs,
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 250, in GetVibsAndMetaFromSources
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     vibsToInstall = Transaction._getVibsFromUrls(vibUrls)
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:   File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 905, in _getVibsFromUrls
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR:     raise Errors.VibDownloadError(url, None, str(e))
2024-11-13T07:03:07Z esxupdate: 5076018: esxupdate: ERROR: vmware.esximage.Errors.VibDownloadError: ('https://VR:443/vib/vmware-hbr-agent.vib', None, "('https://VR:443/vib/vmware-hbr-agent.vib', '', '<urlopen error _ssl.c:1114: The handshake operation timed out>')")

Environment

VCenter : 7.0.3 

vSphere replication: 9.0.2 

 

Cause

The inability to retrieve certificate details further suggests that there is a network issue or configuration problem with VR connectivity on port 443, preventing the handshake from completing.

Resolution

1. Validate Port 443 Connectivity ( (Intra-Site Validation)

  Confirm that port 443 is open between the ESXi host and the vSphere Replication appliance on the same site.

    nc -zv <vSphere_Replication_IP> 443  

Note: While the above command may indicate a connection, further validation is necessary to ensure proper communication

2. Validate by Installing HBR VIB on ESXi:

On the ESXi host, run the following command to verify communication

   esxcli software vib install -v https://<vSphere_Replication_IP>:443/vib/vmware-hbr-agent.vib  

If the firewall rule between the ESXi host and the vSphere Replication appliance is not properly configured, this command will timeout.

Firewall Rule Validation:

• Review the firewall settings to ensure port 443 is open from the ESXi host to the local vSphere Replication appliance (When it is Legacy replication) 
• Add or update firewall rules as necessary to allow uninterrupted communication on port 443.

Additional Information

Run this command in vSphere Replication to confirm it is good

LD_LIBRARY_PATH=/usr/lib /usr/bin/wget --no-check-certificate https://VSpherereplication IP:443/vib/vmware-hbr-agent.vib

 

admin@dcvsra [ ~ ]$ LD_LIBRARY_PATH=/usr/lib /usr/bin/wget --no-check-certificate https://VSpherereplication:443/vib/vmware-hbr-agent.vib        

                                                                                                --2024-11-19 10:49:41--  https://VR/vib/vmware-hbr-agent.vib
Connecting to VSphere replication IP:443... connected.
WARNING: cannot verify VSphere replication's certificate, issued by ‘C=US,ST=California,L=Palo Alto,OU=SRM,O=VMware\\, Inc.’:
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 27407470 (26M) [text/plain]
Saving to: ‘vmware-hbr-agent.vib.1’

vmware-hbr-agent.vib.1                                      100%[========================================================================================================================================>]  26.14M  --.-KB/s    in 0.05s