"Insufficient resources to fail over this virtual machine" error when vSphere HA triggers a failover for VMs on a not-responding ESXi host

Products

• VMware vSphere ESXi

Issue/Introduction

• vSphere HA triggers a failover for all the VMs on a host that is in not-responding state, but the task fails for some VMs with the below error:

     Insufficient resources to fail over this virtual machine. vSphere HA will retry the fail over when enough resources are available. Reason: Unable to find healthy compatible hosts for the VM.

• From /var/run/log/fdm.log of the Primary Host:

YYYY-MM-DDTHH:MM:SS Db(167) Fdm[2102515]: [Originator@6876 sub=Placement opID=29fd####] Vm /vmfs/volumes/65fa30ef-11bc16a6-da4c-############/<VM_Name>/<VM_Name>.vmx failed placement with fault [N3Vim5Fault21NoActiveHostInClusterE:0x000000f2a38####
YYYY-MM-DDTHH:MM:SS Db(167) Fdm[2102515]: [Originator@6876 sub=Placement opID=29fd####] Setting insufficient resource timeout to 60 seconds and retrying placement for vm /vmfs/volumes/65fa30ef-11bc16a6-da4c-############/<VM_Name>/<VM_Name>.vmx
YYYY-MM-DDTHH:MM:SS Db(167) Fdm[2102515]: [Originator@6876 sub=Placement opID=29fd####] Reset the placement status of vm /vmfs/volumes/65fa30ef-11bc16a6-da4c-############/<VM_Name>/<VM_Name>.vmx to 0, vim.fault.NoActiveHostInCluster (vim.fault.NoActiveHostInCluster)

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x
VMware vCenter Server 7.x
VMware vCenter Server 8.x

Cause

• This issue is observed when the problematic VMs have an ISO file attached from a datastore that is not accessible by other ESXi hosts in the cluster.
• In such cases, no other ESXi hosts are compatible for the HA Failover of the VMs as they do not have access to the ISO file.
• It can also occur if manual intervention is performed by means of migrating VMs or placing hosts into Maintenance Mode whilst the cluster's inventory is being re-organized.

Resolution

1. Remove the ISO file attached to the VM.
2. Below is the steps how to remove the ISO file attached to the VM

• 1. Edit the settings of the affected virtual machine.
  2. Locate the CD/DVD drive.
  3. Disconnect any attached ISO files or change the device type to Client Device.
  4. Ensure the Connect at Power On option is deselected if the media is not required for daily operations.
  5. If a datastore containing ISOs is currently inaccessible, either restore access to that datastore from all hosts in the cluster or remove the file backing from the VM configuration.

Additional Information

• There are instances where the datastore with ISO files are in inaccessible state.
• Below error messages in /var/run/log/hostd.log which confirms that the ISO file is inaccessible:
  

  
YYYY-MM-DDTHH:MM:SS In(166) Hostd[20999##]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/65fb84b0-153da07a-2068-############/<VM_Name>/<VM_Name>.vmx] CacheDatastores: fileBacking [] /vmfs/volumes/65f04c41-b644631a-b35a-############/datastorename /<filename>.iso has null datastore reference

  YYYY-MM-DDTHH:MM:SS Db(167) Hostd[20999##]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/65fb84b0-153da07a-2068-############/<VM_Name>/<VM_Name>.vmx] Handling vmx message 1053800: Cannot connect file "/vmfs/volumes/65f04c41-b644631a-b35a-############/datastorename /<filename>.iso" as a CD-ROM image: Could not find the file

• In such cases, remove the Datastore ISO file attached to the VM or attach a valid ISO file.