The vSphere HA agent fails with the error "Applying HA VIBs on the cluster encountered a failure" in the vSphere Client
search cancel

The vSphere HA agent fails with the error "Applying HA VIBs on the cluster encountered a failure" in the vSphere Client

book

Article ID: 399646

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

  • While configuring vSphere HA in the cluster, the following errors appear:

The vSphere HA Agent Unreachable - The vSphere HA Agent on the host cannot be reached
Cannot complete the configuration of the vSphere HA agent on the host. Applying HA VIBs on the cluster encountered failure|
A general system error occurred: Installing HA components failed on the cluster: domain-<ID>.

  • vSphere HA configure task fails with "Cannot complete the configuration of the vSphere HA agent on the host". "Applying HA VIBs on the cluster encountered a failure". "Failed installing HA component on the host: host-XXX".

  • While enabling vSphere HA, the vSphere Lifecycle Manager compliance check subtask on the cluster either hangs at 30% or fails with a timeout error. Additionally, the following messages are logged in /var/run/log/lifecycle.log on the affected ESXi host:

    YYYY-MM-DDTHH:MM:SSZ In(##) lifecycle[########]: imagemanagerctl:### Calling with arguments: components --apply --task-id ####-####-####-####-##### --depot http://<FQDN_of_vCenter>:9084/vum/repository/hostupdate/__micro-depot__vendor-vmw__vmw-ESXi-#.#.#-metadata__index__.xml --depot http://<FQDN_of_vCenter>:9084/vum/repository/hostupdate/__micro-depot__vendor-NTA__metadata-###
    __index__.xml --depot http://FQDN_of_vCenter>:9084/vum/repository/hostupdate/__micro-depot__vendor-vmw__metadata-###__index__.xml --depot http://<FQDN_of_vCenter>:9084/vum/repository/hostupdate/__micro-depot__vendor-vmw__vmw-ESXi-#.#-vmtools-##.#-metadata__index__.xml --component vsphere-fdm:#.#.#-########
    YYYY-MM-DDTHH:MM:SSZWa(##) lifecycle[########]: Downloader:### Download failed: <urlopen error timed out>, 9 retry left...
    YYYY-MM-DDTHH:MM:SSWa(##) lifecycle[########]: Downloader:### Download failed: <urlopen error timed out>, 8 retry left...
    YYYY-MM-DDTHH:MM:SSZ Wa(##) lifecycle[########]: Downloader:### Download failed: <urlopen error timed out>, 7 retry left...

  • The following log messages appear in /var/log/vmware/vpxd/vpxd.log on the vCenter Server:

    YYYY-MM-DDTHH:MM:SS warning vpxd[#####] [Originator@#### sub=Vmomi opID=FdmMonitor-domain-c#####-#####] Got vmacore exception when invoking VMOMI method; <</hgw/host-####>, /fdm>, csi.FdmService.GetAbout, N#Vmacore#Http##HttpExceptionE(HTTP error response: Service Unavailable)
    --> [context]###################################/#######################################/#######################/########################################/################=[/context]
    YYYY-MM-DDThh:mm:ss error vpxd[#####] [Originator@#### sub=HostUpgrader opID=FdmMonitor-domain-c#####-########] Failed to get fdm aboutInfo from host-####: N#Vmomi#Fault##HostCommunication#ExceptionE(Fault cause: vmodl.fault.HostCommunication)

Environment

VMware vCenter Server 8.x
VMware vSphere ESXi 8.x

Cause

This issue occurs because the vSphere Lifecycle Manager (vLCM) / vCenter Update Manager (VUM) port is blocked on all ESXi hosts in the cluster, preventing the communication required for HA component installation.

Resolution

Steps to check if the Update Manager port is blocked:

  1. Establish an SSH connection to the ESXi host as the root user.

  2. Execute the following command: esxcli network firewall ruleset list

  3. Review the output for updateManager. If the status displays as false, the port is currently blocked.

    esxcli network firewall ruleset list
    Name                         Enabled  Enable/Disable configurable  Allowed IP configurable
    ---------------------------  -------  ---------------------------  -----------------------
    CIMHttpServer                  false                        false                     true
    CIMHttpsServer                 false                        false                     true
    CIMSLP                         false                        false                     true
    iSCSI                          false                        false                     true
    vpxHeartbeats                   true                        false                     true
    updateManager                  false                         true                     true
    faultTolerance                  true                         true                     true

How to enable the Update Manager via CLI:

  1. Establish an SSH connection to the ESXi host as the root user.

  2. Execute the following command: esxcli network firewall ruleset set -e true -r updateManager

  3. Confirm the modification by running the command below: esxcli network firewall ruleset list
    Name                         Enabled  Enable/Disable configurable  Allowed IP configurable
    ---------------------------  -------  ---------------------------  -----------------------
    CIMHttpServer                  false                        false                     true
    CIMHttpsServer                 false                        false                     true
    CIMSLP                         false                        false                     true
    iSCSI                          false                        false                     true
    vpxHeartbeats                  true                         false                     true
    updateManager                  true                         true                      true
    faultTolerance                 true                         true                      true


How to enable the vCenter Update Manager firewall rule in the vSphere Client:

  1. In the vSphere Client, select the target ESXi host.

  2. Navigate to the Configure tab, select Firewall, and click Edit.

  3. Locate vCenter Update Manager in the list of rules.

  4. Select the checkbox to enable the port.

  5. Click OK to apply the changes.

Additional Information

  • Validate traffic is routable from the vCenter Server to the ESXi host over TCP ports 80, 443, 902, and 9080 using the following command : curl -v telnet://<ESXi-Host-IP>:<Port-Number>
  • Validate traffic is routable from the ESXi host to the vCenter Server over TCP ports 9084 and 9087 using the following command : nc -zv <VCENTER-IP> <Port-Number>
  • There should also be bidirectional communication between all hosts in the cluster on 8182 TCP and UDP, validate using following command : nc -zv <ESXi-Host-IP> <Port-Number>
  • Refer to the following document for port requirements VMware Ports and Protocols

  • If the previous steps are successful but the 'Connection reset by peer' error persists in the host lifecycle logs, analyze packet captures on both the ESXi host and the vCenter Server to ensure a firewall rule is not resetting the connection.

    • For example: In this case, after a successful 3-way handshake, Instead of receiving the HTTP request, a TCP Reset ([RST, ACK]) is immediately fired from the client's side to vCenter Server, tearing down the connection.

      <Source_IP>  <Destination_IP>   TCP  <Host-Port> -> 9084 [RST, ACK] Seq=1 Ack=1 Win=xxx Len=0
  • If a timeout is observed from the DNS Server to the ESXi host during testing, it indicates a potential host-level misconfiguration of the DNS service. To resolve this, clear the DNS configuration on the affected ESXi host and then reconfigure the settings.
Powered by Wolken Software