Unable to power on a virtual machine after cancelling snapshot consolidation

Products

VMware vSphere ESXi

Issue/Introduction

The following symptoms may be observed:

Snapshot consolidation was initiated while the virtual machine was powered off.
The virtual machine is unable to power on after the consolidation process was cancelled.

Attempting to power on the VM fails with an error message similar to:

Failed to open '/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmdk' with flags 0x13 The system cannot find the file specified (25).

In the /var/run/log/hostd.log file, entries similar to the following may be recorded:

YYYY-MM-DDThh:mm:ss.zzzZ info hostd[29EC2B70] [Originator@6876 sub=Libs opID=c4c8aa4f-da14-4209-a243-5e687f786047-162836-ngc-a1-72-1b7d user=vpxuser:Domain\User] SnapshotVigorConsolidate: Failed to consolidate: Operation was canceled (5)
.
.
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 3340 : Virtual machine Virtual_Machine_Name disks consolidation failed on Cluster_Name in cluster Cluster_Name. in ha-datacenter.
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 3341 : Virtual machine Virtual_Machine_Name disks consolidation is needed on Cluster_Name in cluster Cluster_Name. in ha-datacenter.
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmx] Send config update invoked
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=Libs] OBJLIB-FILEBE : FileBEOpen: can't open '/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmdk' : Could not find the file (393218).
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=DiskLib] DISKLIB-DSCPTR: DescriptorOpenInt: failed to open '/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmdk': Could not find the file (60002)
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=DiskLib] DISKLIB-LINK : "/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmdk" : failed to open (The system cannot find the file specified).
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=DiskLib] DISKLIB-CHAIN : "/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmdk" : failed to open (The system cannot find the file specified).
YYYY-MM-DDThh:mm:ss.zzzZ info hostd[28EC2B70] [Originator@6876 sub=DiskLib] DISKLIB-LIB : Failed to open '/vmfs/volumes/datastore/Virtual_Machine_Folder/Virtual_Machine.vmdk' with flags 0x13 The system cannot find the file specified (25).

Environment

VMware vSphere ESXi 6.5

Cause

In some older versions of VMware ESXi, the snapshot virtual disk consolidation and snapshot deletion processes can be cancelled. This is a known issue affecting VMware ESXi 6.5.

These processes should never be cancelled under any circumstances.
Cancelling them carries a high risk of virtual disk corruption.

Resolution

To resolve the issue, perform the following steps:

Restore the virtual machine (VM) from a backup taken prior to the failed consolidation attempt.
Update ESXi to a version that prevents consolidations from being cancelled.

- VMware ESXi 6.5 U1 (Build: 5310538) or above

Note: Even after upgrading, the user interface may still provide the option to cancel these processes. Do not attempt to cancel consolidation or snapshot deletion at any time.

Workaround:

If restoring from backup is not possible, a workaround is to repoint the VM to a parent virtual disk.

If the parent virtual disk is not corrupted, the VM may be able to power on.

Warning: This process will cause all data in the snapshot virtual disks to be lost. For example, if a snapshot was active for two months, two months of data will be lost. Ensure that valid backups of the VM exist before proceeding.

Steps to Repoint the VM to a Parent Virtual Disk:

Open the VM’s .vmx file and locate the SCSI disk entry that failed to open.
Open an SSH session to the ESXi host where the VM resides.
Navigate to the VM’s directory:
```
cd /vmfs/volumes/datastoreName/VMname/
```
Edit the VM configuration file:
```
vi <VMname>.vmx
```
Look for the failed disk entry (e.g., scsi0:0.fileName = "TestVM-000008.vmdk").
Update the entry to point to the parent virtual disk (e.g., TestVM-000007.vmdk).
Power on the virtual machine.

- If the VM still fails to power on, repeat the procedure for the parent snapshot disk and try again.
- It is common for multiple snapshot virtual disks to become corrupted in such cases. The procedure may need to be repeated several times until a valid parent disk in the chain is identified.

Additional Information

Failed to power on virtual machine

Impact/Risks:
Repointing a virtual machine to use a parent virtual disk will cause the data contained in the child virtual disk to be lost.