"Insufficient resources to fail over this virtual machine" error when vSphere HA triggers a failover for VMs on a not-responding ESXi host

Products

• VMware vSphere ESXi

Issue/Introduction

• vSphere HA triggers a failover for all the VMs on a host that is in not-responding state, but the task fails for some VMs with the below error:

     Insufficient resources to fail over this virtual machine. vSphere HA will retry the fail over when enough resources are available. Reason: Unable to find healthy compatible hosts for the VM.

• From /var/run/log/fdm.log of the Primary Host:

YYYY-MM-DDTHH:MM:SS Db(167) Fdm[2102515]: [Originator@6876 sub=Placement opID=29fd####] Vm /vmfs/volumes/65fa30ef-11bc16a6-da4c-############/<VM_Name>/<VM_Name>.vmx failed placement with fault [N3Vim5Fault21NoActiveHostInClusterE:0x000000f2a38####
YYYY-MM-DDTHH:MM:SS Db(167) Fdm[2102515]: [Originator@6876 sub=Placement opID=29fd####] Setting insufficient resource timeout to 60 seconds and retrying placement for vm /vmfs/volumes/65fa30ef-11bc16a6-da4c-############/<VM_Name>/<VM_Name>.vmx
YYYY-MM-DDTHH:MM:SS Db(167) Fdm[2102515]: [Originator@6876 sub=Placement opID=29fd####] Reset the placement status of vm /vmfs/volumes/65fa30ef-11bc16a6-da4c-############/<VM_Name>/<VM_Name>.vmx to 0, vim.fault.NoActiveHostInCluster (vim.fault.NoActiveHostInCluster)

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x
VMware vCenter Server 7.x
VMware vCenter Server 8.x

Cause

• This issue is observed when the problematic VMs have an ISO file attached from a datastore that is not accessible by other ESXi hosts in the cluster.
• In such cases, no other ESXi hosts are compatible for the HA Failover of the VMs as they do not have access to the ISO file.
• It can also occur if manual intervention is performed by means of migrating VMs or placing hosts into Maintenance Mode whilst the cluster's inventory is being re-organized.

Resolution

• Remove the ISO file attached to the VM.

Additional Information

• There are instances where the datastore with ISO files are in inaccessible state.
• Below error messages in /var/run/log/hostd.log which confirms that the ISO file is inaccessible:
  

  
YYYY-MM-DDTHH:MM:SS In(166) Hostd[20999##]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/65fb84b0-153da07a-2068-############/<VM_Name>/<VM_Name>.vmx] CacheDatastores: fileBacking [] /vmfs/volumes/65f04c41-b644631a-b35a-############/datastorename /<filename>.iso has null datastore reference

  YYYY-MM-DDTHH:MM:SS Db(167) Hostd[20999##]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/65fb84b0-153da07a-2068-############/<VM_Name>/<VM_Name>.vmx] Handling vmx message 1053800: Cannot connect file "/vmfs/volumes/65f04c41-b644631a-b35a-############/datastorename /<filename>.iso" as a CD-ROM image: Could not find the file

• In such cases, remove the Datastore ISO file attached to the VM or attach a valid ISO file.