VM deletion fails due to CTK files issue

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• Virtual machine 'Delete from disk' operation fails with error 'device or resource busy'.
• Following warning can be observed for the virtual machine:

'Virtual machine disks consolidation needed.'

• Based on the consolidation warning, an attempt to perform VM consolidation results in error:

'Disk consolidation for VM <vm_name> has failed: Unable to access file since its locked.'

• Additionally, deleting VM files (specifically *.vmdk files) as part of manual clean up of the VM fails with the following error:

'Cannot delete files '/vmfs/volumes/xxxxxxxx-datastore-uuid-xxxxxxxxxxx/VMName/VMName-00000X.vmdk.'
'The virtual disk is either corrupt or not a supported format.'

Environment

• vCenter 7.x
• vCenter 8..x
• vCenter 9.x
• ESXi 7.x
• ESXi 8.x
• ESX 9.x

Cause

The deletion of VM or the associated vmdk files fail due to old CTK files in the virtual machine folder path '/vmfs/volumes/xxxxxxxx-datastore-uuid-xxxxxxxxxxx/VMName/'

Resolution

1. Confirm that the files in the VM folder are not in use or locked by any process. To resolve file lock issue refer: Investigating Virtual Machine file locks on ESXi Host(s)
2. After ensuring VM file is not locked, proceed to move all the 'ctk' files to a temporary sub folder:

   • Connect to the ESXi host on which the VM resides using SSH client.

   • Navigate to the VM folder:
       cd /vmfs/volumes/xxxxxxxx-datastore-uuid-xxxxxxxxxxx/<VMName>
   • Create a temporary folder, inside the VM folder, to move the *.ctk files: 
        mkdir tmpctk
   • Move all the CTK files to this folder:
        mv *ctk.vmdk tmpctk/
3. From the vCenter, delete the VM(Delete from Disk). Or, if performing a manual clean up of VM files, delete the *.vmdk files which were failing previously.
4. Delete the previously moved 'ctk' files and the folder 'tmpctk' from the VM folder.

Additional Information

The issue of corrupt CTK files can cause issue with VM snapshot operations: Error: "the virtual disk is either corrupted or not a supported format" during snapshot operation

For more information about Changed Block Tracking see: Changed Block Tracking (CBT) on virtual machines