Failed to lock the file or One or more disks are busy deleting VM snapshots

Products

VMware vSphere ESXi

Issue/Introduction

Virtual Machine file lock have multiple reasons. This article covers the file lock caused by backup application proxy virtual machine.

For more information about Virtual Machine file lock, see Investigating virtual machine file locks on ESXi hosts (10051).

Symptoms:

Snapshots cannot be committed.
Snapshot commits without errors and Snapshot Manager is no longer populated. However, the snapshot disk is present in the virtual machine directory. (the VM is still running on Snapshot)
Consolidation failed with the errors similar to:

Failed to lock the file or One or more disks are busy.

or

Unable to consolidate virtual machine snapshots due to file lock.

or

Unable to access file since it is locked.
The virtual machine summary tab display messages similar to:

snapshot consolidation required.

or

Virtual machine disks consolidation is needed.

or

Virtual machine Consolidation Needed status.

Note: For additional symptoms and log entries, see the Related Information section.

Environment

VMware vSphere ESXi 6.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

This issue occurs if the backup proxy virtual machine or other virtual machine holds a lock on the base disk or previous snapshot file of the virtual machine that has the consolidation issue, preventing the snapshot consolidation to succeed.

Resolution

To resolve this issue, remove the virtual machine disk(s) from backup proxy VM.

Caution: Make sure there is no backup job running on the virtual machine that has the consolidation issue.

Right-click the backup proxy virtual machine.
Click Edit Settings.
Expand all the Hard Disk(s).
Select the Hard Disk(s) which belong to the virtual machine that has the problem.
Click on the X beside the Hard Disk to unmount the Hard Disk from the VM.
Caution: Do NOT select Delete files from the datastore.
Click OK.
Consolidate/Delete the snapshot on the VM.

Additional Information

VMware Skyline Health Diagnostics for vSphere - FAQ

You experience these additional symptoms:

In the vmware.log, you see errors similar to:

vmx| ConsolidateOnlineCB: nextState = 2 uid 3 vmx| Foundry operation failed with system error: Device or resource busy (16), translated to 5 vmx| ConsolidateOnlineCB: Done with consolidate
When you attempt to remove the datastore, you see this error:

The resource '<VMFS-UUID>' is in use.
If you are running third party back up software, consolidation might fail with the following errors in vmware.log file:

vcpu-0| Vix: [8803 mainDispatch.c:4084]: VMAutomation_ReportPowerOpFinished: statevar=3, newAppState=1881, success=1 additionalError=0
vcpu-0| Vix: [8803 vigorCommands.c:577]: VigorSnapshotManagerConsolidateCallback: snapshotErr = Failed to lock the file (5:4008)
vcpu-0| SnapshotVMXConsolidateOnlineCB: Destroying thread 6
vcpu-0| Turning off snapshot info cache.
vcpu-0| Turning off snapshot disk cache.
vcpu-0| SnapshotVMXConsolidateOnlineCB: Done with consolidate
In the vmkernel or messages log files, you see entries similar to:

vmkernel: gen 2141, mode 1, owner 4b94bb81-XXXXXXXX-3bd1-XXXXXXXXXXX mtime 244622]on volume 'LUN03'. vmkernel: [YYYY-MM-DDTHH:MM:SS] cpu2:4109)FS3: 2890: [Requested mode: 1] Lock [type 10c00001 offset 7505920 v 920, hb offset 3510272 vmkernel: gen 2141, mode 1, owner 4b94bb81-XXXXXXXX-3bd1-XXXXXXXXXXX mtime 244622] is not free on volume 'LUN03' vmkernel: [YYYY-MM-DDTHH:MM:SS] cpu2:4111)FS3: 2798: [Requested mode: 1] Checking liveness of lock holders [type 10c00001 offset 7313408 v 796, hb offset 3510272
In the hostd.log file, you see entries similar to these during the snapshot delete process:

DISKLIB-LIB : Failed to delete disk '/vmfs/volumes/4c5f4b7a-XXXXXXXX-32ad-XXXXXXXXX/TESTVM/TESTVM_1-000001.vmdk' or one of its components: Device or resource busy
When you attempt to consolidate by right-clicking the virtual machine and clicking Snapshot > Consolidate, you see errors similar to:

Consolidate virtual machine disk files <hostname> Unable to access file <unspecified filename> since it is locked
Consolidation failed for disk node 'scsi0:8': msg.fileio.lock.

Consolidating/Committing snapshots in VMware ESXi
Restarting the Management agents in ESXi
Determining if a virtual disk is attached to another virtual machine
FAQ: VMware vSphere Storage APIs – Data Protection (formerly known as VMware vStorage APIs for Data Protection or VADP)
Consolidating disks associated with a backup snapshot fails with the error: The file is already in use
"An error occurred while consolidating disks:" deleting one or more snapshots or consolidate disks