vSphere HA connection fails with error "HA Agent Unreachable" & "Uninitialized"

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• vSphere HA or high availability connection issues after vCenter upgrade.
• HA state reported to be "HA Agent Unreachable"
• HA state also shows Uninitialized error occurred during remediation.
• FDM VIB installation operation fails with error "Cannot open volume".
• Storage devices may report as read only.

 

Validation:

1. HA Unreachable & Uninitialized. You may observe this in vCenter UI>Inventory>Cluster>Hosts

 

 

 

2. FDM VIB Installation fail while running the installation command on Host cli.

 

Hostname:/tmp] esxcli software vib install -v /tmp/VMware_bootbank_vmware-fdm_7.0.3-24321951.vib -f

[InstallationError]
Failed to query file system stats: Errors:
Error getting data for filesystem on '/vmfs/volumes/                   ': Cannot open volume: /vmfs/volumes/               skipping

 

 

3.Storage devices may report read only in the /var/run/log/vmkernel.log with below sense code.

0x7 - Data protect
0x27 -Write protected

2025-03-08T04:12:05.656Z cpu54:2098323)ScsiDeviceIO: 4176: Cmd(0x45d9c7ec9608)0x2a, CmdSN 0x3b from world 44176919 to dev "naa.6000##############44" failed H: 0x0 D:0x2 P: 0x0 Valid sense data: 0x7 0x27 0x0

 

Environment

VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 8.0.x

Cause

Devices which were failing with the error "Cannot open volume" are reported be Lun which was detected as snapshot. Devices report as read only. This is expected at the peer site when using the array-based replication. So, when using array-based replication and during array fail over operation, Source devices are made read-only and optionally takes a snapshot of the source devices in anticipation of a fail over.

During the FDM VIB installation, Datastore reported with "Lun is detected to be snapshot" are being queried and since it not mounted and read-only the installation operation fails as they cannot open the volume. You may see observe this under vCenter UI>Inventory>Cluster>Hosts>Datastore

 

You may run the command "localcli storage vmfs snapshot list" to check the datastores which is reported as snapshot. Since Datastore which was detected as snapshot are mounted on other hosts, you may see the mountability has true.

Volune Name:  #########
VMFS WID:  #####
Con mount: true
Reason for un-mountability:
Can resignature: false
Reason for non-resignaturability: the volume is being actively used
Unresolved Extent Count: 1

Resolution

Devices report as read only. This is expected at the peer site when using the array-based replication. These Snap devices need not be mounted, and they will have to be cleaned UP. Storage vendor is aware of this and for cleanup recommendation is to go for global setting on SRA.

Engage the SRA vendor to modify one of the SRA global options. This should change the SRA behavior and clean up the devices after testing.