Enabling HA on a cluster fails with error - Cannot complete the configuration of the vSphere HA agent on the host. "Applying HA VIBs on the cluster encountered a failure"

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• The vCenter UI shows the following error when attempting to enable HA on a cluster: 

Cannot complete the configuration of the vSphere HA agent on the host. "Applying HA VIBs on the cluster encountered a failure"

• From the /var/run/log/fdm.log file on the ESXi host, the following entries will be seen with regards to the LUN ########77####59-0666-############  : 

Originator@#### sub=Invt opID=WorkQueue-########] Datastore (/vmfs/volumes/########77####59-0666-############) is inaccessible to localhost
Originator@#### sub=Invt opID=WorkQueue-########] Datastore (/vmfs/volumes/########77####59-0666-############) has inaccessibleReason for localhost _.
Originator@#### sub=Policy opID=WorkQueuc-######] Add inaccessible datastore ########77####59-0666-############ to inaccessibleDs.
Originator@#### sub=Policy opID=WorkQueue-######] Datastore ########-774b4259-0666-############ has no affected VMs
Originator@#### sub=Cluster opID-clusterManager. cpp: 983-#########] Local datastore /vmfs/volumes/########77####59-0666-############ accessibility change: true => false
Originator@#### sub=Invt opID=clusterManager. cpp: 983-#########] Notify datastore (/vmfs/volumes/########77####59-0666-############)

Cause

• Inconsistent LUN presentation: The LUN is not presented to all hosts in the cluster.
• LUN not visible to the host: The ESXi host cannot see the LUN due to Fibre Channel or iSCSI connectivity issues, such as incorrect zoning, incorrect iSCSI configuration, or physical cabling problems.
• LUN set to read-only: The LUN is marked as read-only on the storage array level, preventing the HA agent from writing its necessary configuration or heartbeat files.
• Administratively detached LUN: The LUN was previously detached manually and needs to be manually re-activated with esxcli commands.

Resolution

• The following KB article has the steps to troubleshoot the LUN connectivity issues on ESXi hosts.
• The following KB article provides steps to unmount a LUN from an ESXi host, which includes unmounting the file system and detaching the datastore/storage device if the LUN is not being used in the environment. These steps must be performed for each ESXi host.
• After unmounting the LUN in all ESX Hosts where it is no longer in use the vSphere HA configuration agent will be available.
• Open a Broadcom support request with the storage team to troubleshoot the LUN connectivity issues on ESXi hosts to understand the issue further.