Backups failing for some of the VMs in one of the host

Products

• VMware vSphere ESX 7.x
• VMware vCenter Server 7.0

Issue/Introduction

Symptoms :-

• Backups failing for some of the VMs in one of the host.
• Some of the VMs in the host are having snapshot creation issues while initiating the backup from backup tool.
• VM’s Disks show as 0kb
• This issue is seen when some of the affected VMs is vMotioned (by DRS) on to this host.
• The issue stands resolved and VM registers their disks properly after a power off & on of the VM in the same host or a vMotion to another host

Environment

• VMware vCenter Server 7.x
• VMware vSphere ESXi 7.x
  

Cause

Storage is not accessible and the volumes show as unmounted under the host client though they show normal under the vCenter.

From the host's logs in the /var/run/log/hostd.log, we see the following warnings:-

warning hostd[3215759] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmx] PopulateCache failed: _diskAccess : true, _storageAccessible : false
warning hostd[3215759] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmdk] Disk properties cache miss for disk path: /vmfs/volumes/[Datastore]/VM/VM.vmdk.
warning hostd[3215759] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmx] UpdateStorageAccessibilityStatusInt: The datastore [Datastore-Name] is not accessible
info hostd[2100331] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmx] Snapshot tree refresh ignored because storage is not accessible.
warning hostd[3215759] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmx] File - failed to get objectId, '/vmfs/volumes/[Datastore]/VM/VM.vmx': Operation not supported (11)
warning hostd[3215759] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmx] Snapshot create succeeded, but UID 1680 not in list.
warning hostd[3215759] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/[Datastore]/VM/VM.vmx] Failed operation

 

Resolution

To make sure all the storage is accessible again and reconfigure the volumes for the host

1) Login to the ESXi host client
2) Select Storage
3) Right click the datastore
4) Mount Datastore

Mount Datastores