An error occurred during host configuration - Host fails to download vmware-hbr-agent VIB (Temporary failure in name resolution)

Products

• VMware Live Recovery

Issue/Introduction

Symptoms:

1. Host fails to download vmware-hbr-agent.vib

2. Host constantly attempts to install vmware-hbr-agent.vib

You see the below tasks getting logged in ESXi - 

Install
esxi01.vmware.com
An error occurred during host configuration.
[e-mail]
10/05/2023, 1:01:24 PM
10/05/2023, 1:02:03 PM

Task Name: Install
Status   : An error occurred during host configuration: /usr/sbin/esxupdate returned with exit status: 7.
Initiator: [e-mail]
Target   : Host 192.X.X.X
Server   : vCenter vcsa.vmware.com

vpxa.log :

2023-10-05T17:04:43.407Z info vpxa[2100734] [Originator@6876 sub=Default opID=71ab0cc4-ba6d-4222-a2a8-24c3c9002a57-HMSINT-29-72-65] [VpxLRO] -- ERROR task-41013 -- patchManager -- vim.host.PatchManager.InstallV2: vim.fault.PlatformConfigFault:
--> Result:
--> (vim.fault.PlatformConfigFault) {
-->  faultCause = (vmodl.MethodFault) null,
-->  faultMessage = <unset>,
-->  text = "/usr/sbin/esxupdate returned with exit status: 7"
-->  msg = "An error occurred during host configuration."
--> }
--> Args:
-->
--> Arg metaUrls:
--> (string) []
--> Arg bundleUrls:
--> (string) []
--> Arg vibUrls:
--> (string) [
-->  "https://vSphere-replication.vmware.com:8043/vib/vmware-hbr-agent.vib"
--> ]
--> Arg spec:
--> (vim.host.PatchManager.PatchManagerOperationSpec) {
-->  proxy = "http://vcenter.vmware.com",
-->  port = 80,
-->  userName = <unset>,
-->  password = <unset>,
-->  cmdOption = <unset>
--> }

vpxd.log :

2024-03-19T17:02:56.467Z info vpxd[07995] [Originator@6876 sub=Default opID=c2fb10d3-695c-466c-ab47-d1f67ae7c8a2-HMSINT-46-18] [VpxLRO] -- ERROR task-23603 -- patchManager-10 -- vim.host.PatchManager.InstallV2: vim.fault.PlatformConfigFault:
--> Result:
--> (vim.fault.PlatformConfigFault) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    text = "/usr/sbin/esxupdate returned with exit status: 7"
-->    msg = "An error occurred during host configuration."
--> }
--> Args:
-->
--> Arg metaUrls:
--> (string) []
--> Arg bundleUrls:
--> (string) []
--> Arg vibUrls:
--> (string) [
-->    "https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib"
--> ]
--> Arg spec:
--> (vim.host.PatchManager.PatchManagerOperationSpec) {
-->    proxy = "http://vCenter.vmware.com",
-->    port = 80,
-->    userName = <unset>,
-->    password = <unset>,
-->    cmdOption = <unset>
--> }
 

esxupdate.log :

An error occurred during host configuration: /usr/sbin/esxupdate returned with exit status: 7.

2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: esximage.Downloader.DownloaderError: ('https://192.168.1.2:8043/vib/vmware-hbr-agent.vib', '', '<urlopen error [Errno -3] Temporary failure in name resolution>')
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: During handling of the above exception, another exception occurred:
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: Traceback (most recent call last):
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: File "/usr/sbin/esxupdate", line 378, in main
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: cmd.Run()
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: File "/lib64/python3.8/site-packages/vmware/esx5update/Cmdline.py", line 161, in Run
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: res = t.InstallVibsFromSources(viburls, metaurls, None,
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 834, in InstallVibsFromSources
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: self.GetVibsAndMetaFromSources(vibUrls, metaUrls, vibspecs,
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 269, in GetVibsAndMetaFromSources
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: vibsToInstall = Transaction._getVibsFromUrls(vibUrls)
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: File "/lib64/python3.8/site-packages/vmware/esximage/Transaction.py", line 928, in _getVibsFromUrls
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: raise Errors.VibDownloadError(url, None, str(e))
2023-11-23T15:47:12Z Er(11) esxupdate[8059525]: esximage.Errors.VibDownloadError: ('https://192.168.1.2:8043/vib/vmware-hbr-agent.vib', None, "('https://192.168.1.2:8043/vib/vmware-hbr-agent.vib', '', '<urlopen error [Errno -3] Temporary failure in name resolution
 

2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: --- Command: update Args: ['update'] Options:
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: viburls = ['https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib']
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: meta = None
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: hamode = True
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: proxyurl = http://vCenter.vmware.com:80
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: timeout = 30.0
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: retry = 5
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: loglevel = None
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: cachesize = None
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: cleancache = None
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: maintenancemode = None
2024-03-18T14:28:15Z esxupdate: 2405325: esxupdate: INFO: nosigcheck = None
2024-03-18T14:28:15Z esxupdate: 2405325: HostImage: INFO: Installers initiated are {'live': <vmware.esximage.Installer.LiveImageInstaller.LiveImageInstaller object at 0xcacbf863a0>, 'boot': <vmware.esximage.Installer.BootBankInstaller.BootBankInstaller object at 0xcacc0af370>, 'locker': <vmware.esximage.Installer.LockerInstaller.LockerInstaller object at 0xcacbf866d0>}
2024-03-18T14:28:15Z esxupdate: 2405325: imageprofile: INFO: Adding VIB VMware_locker_tools-light_12.1.5.20735119-21422485 to ImageProfile ESXi-7.0U3n-21930508-standard
2024-03-18T14:28:15Z esxupdate: 2405325: imageprofile: DEBUG: Adding Component VMware-VM-Tools_12.1.5.20735119-21422485 to ImageProfile ESXi-7.0U3n-21930508-standard
2024-03-18T14:28:15Z esxupdate: 2405325: imageprofile: DEBUG: Removing reserved Component VMware-VM-Tools_12.1.5.20735119-21422485 in ImageProfile ESXi-7.0U3n-21930508-standard
2024-03-18T14:28:16Z esxupdate: 2405325: downloader: INFO: Opening https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib for download
2024-03-18T14:28:16Z esxupdate: 2405325: downloader: INFO: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-03-18T14:28:16Z esxupdate: 2405325: Proxy: ERROR: Proxy server cannot connect to the endpoint: HTTP/1.1 404 Not Found^M Date: Mon, 18 Mar 2024 14:28:16 GMT^M Connection: close^M Content-Type: text/html^M Content-Length: 48^M ^M <
2024-03-18T14:28:19Z esxupdate: 2405325: downloader: WARNING: Download failed: Proxy server vCenter.vmware.com:80 returned status 404, 4 retry left...
2024-03-18T14:28:19Z esxupdate: 2405325: downloader: INFO: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-03-18T14:28:19Z esxupdate: 2405325: Proxy: ERROR: Proxy server cannot connect to the endpoint: HTTP/1.1 404 Not Found^M Date: Mon, 18 Mar 2024 14:28:19 GMT^M Connection: close^M Content-Type: text/html^M Content-Length: 48^M ^M <
2024-03-18T14:28:22Z esxupdate: 2405325: downloader: WARNING: Download failed: Proxy server vCenter.vmware.com:80 returned status 404, 3 retry left...
2024-03-18T14:28:22Z esxupdate: 2405325: downloader: INFO: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-03-18T14:28:22Z esxupdate: 2405325: Proxy: ERROR: Proxy server cannot connect to the endpoint: HTTP/1.1 404 Not Found^M Date: Mon, 18 Mar 2024 14:28:22 GMT^M Connection: close^M Content-Type: text/html^M Content-Length: 48^M ^M <
2024-03-18T14:28:28Z esxupdate: 2405325: esxupdate: ERROR: vmware.esximage.Errors.VibDownloadError: ('https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib', None, "('https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib', '', 'Proxy server vCenter.vmware.com:80 returned status 404')")

2024-04-01T15:53:32Z In(14) esxupdate[2370112]: Opening https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib for download
2024-04-01T15:53:32Z In(14) esxupdate[2370112]: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-04-01T15:53:35Z Wa(12) esxupdate[2370112]: Download failed: <urlopen error [Errno -2] Name or service not known>, 4 retry left...
2024-04-01T15:53:35Z In(14) esxupdate[2370112]: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-04-01T15:53:38Z Wa(12) esxupdate[2370112]: Download failed: <urlopen error [Errno -2] Name or service not known>, 3 retry left...
2024-04-01T15:53:38Z In(14) esxupdate[2370112]: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-04-01T15:53:41Z Wa(12) esxupdate[2370112]: Download failed: <urlopen error [Errno -2] Name or service not known>, 2 retry left...
2024-04-01T15:53:41Z In(14) esxupdate[2370112]: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}
2024-04-01T15:53:44Z Wa(12) esxupdate[2370112]: Download failed: <urlopen error [Errno -2] Name or service not known>, 1 retry left...
2024-04-01T15:53:44Z In(14) esxupdate[2370112]: Proxies configured: {'http': 'http://vCenter.vmware.com:80', 'https': 'http://vCenter.vmware.com:80', 'ftp': 'http://vCenter.vmware.com:80'}

2024-04-01T15:53:44Z Er(11) esxupdate[2370112]: An esxupdate error exception was caught:
ho
2024-04-01T15:53:44Z Er(11) esxupdate[2370112]: raise Errors.VibDownloadError(url, None, str(e))
2024-04-01T15:53:44Z Er(11) esxupdate[2370112]: esximage.Errors.VibDownloadError: ('https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib', None, "('https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib', '', '<urlopen error [Errno -2] Name or service not known>')")
 

Environment

VMware vSphere Replication 8.x

Cause

DNS configuration is incorrect or not configured at all. Network ports are blocked on the firewall.

Resolution

 

Check if the VIB is installed 

[root@ESXihost:~] esxcli software vib list | grep -i hbr
vmware-hbr-agent               8.0.2-0.0.22517424                     VMware  VMwareCertified   2024-03-21
 

Remove the VIB from the problematic host 

[root@ESXihost:~] esxcli software vib remove -n vmware-hbr-agent
Removal Result
   Message: Operation finished successfully.
   Reboot Required: false
   VIBs Installed:
   VIBs Removed: VMware_bootbank_vmware-hbr-agent_8.0.2-0.0.22517424
   VIBs Skipped:
 

Try installing the VIB manually and check the output 

Disable the firewall before performing the this step and enable it after its complete. 

esxcli network firewall set --enabled false
esxcli network firewall set --enabled true (Set it back to true after downloading the file)
 

Command to install the VIB manually 

[root@ESXihost:~] esxcli software vib install --proxy=http://vCenter.vmware.com:80 -v https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib
 [VibDownloadError]
 ('https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib', '', 'Proxy server vCenter.vmware.com:80 returned status 404')
        url = https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib
Please refer to the log file for more details. (Refer to esxupdate.log)

Check if you can download the VIB from a browser without any errors - https://replication.vmware.com:8043/vib/vmware-hbr-agent.vib
 

Ensure that there isn't more than 1 node in the vCenter inventory having the same hostname. 

Bad Example:

[root@vCenter:~] hostname
vCenter.vmware.com

[root@Atlanta:~] hostname -i
172.X.X.X

root@ESXihost [ ~ ]# hostname
vCenter.vmware.com

root@ESXihost [ ~ ]# hostname -i
127.0.0.1

The host and vCenter have the same hostnames - vCenter.vmware.com

[root@localhost:~] hostname
localhost.localdomain

[root@localhost:~] hostname -i
127.0.0.1

Here, there is no DNS record created for this host. 

1. Make sure vCenter, vSphere Replication, SRM and ESXi hosts are using the correct DNS server. (Decommissioned DNS must not be used)

2. Create forward and reverse lookup records in DNS for vCenter, vSphere Replication, SRM and ESXi hosts, if it does not exist. 

3. If step 1 & 2 is not working, update the hosts file of vCenter, vSphere Replication and/or SRM in the way mentioned below. 

Update the hosts file at "/etc/hosts" file in the format of "IP FQDN Hostname"; Example: "192.168.11.20 SRM01.vmware.local SRM01"


 

1. Verify the DNS record for the host is correctly configured in DNS manager (forward/reverse lookup must be configured)

2. Verify the hostname in DCUI matches the DNS record 

3. Verify the Primary & alternate DNS servers are correct & working from the host.

4. Verfiy the custom DNS suffix is set