Unable to cleanup files from VM after it became corrupt

Products

• VMware vSphere ESXi

Issue/Introduction

• The VM was deleted from disk already.
• Unable to remove some of the old .vmdk files as they are reported locked. 
• Cannot delete files from datastore.
• Running the command <vmfsfilelockinfo -p <.lck file>  shows a lock is still being held by HOST1
  

  Looking for lock owners on ".########-####-####-####-9440c98d7538.lck"
  ".########-####-####-####-9440c98d7538.lck" is locked in Read-Only mode by host having mac address ['##:##:##:##:##:##']
  Trying to make use of Fault Domain Manager
  ----------------------------------------------------------------------
  Found 4 ESX hosts using Fault Domain Manager.
  ----------------------------------------------------------------------
  Searching on Host <HOST3>
  Searching on Host <HOST4>
  Searching on Host <HOST2>
  Searching on Host <HOST1>
  	MAC Address : ##:##:##:##:##:##
   
   
  Host owning the lock on file is <HOST1>, lockMode : Read-Only

Environment

VMware vSAN

VMware vCenter

 

Cause

• The file is being locked by some other VM process.
• In this case Veeam Proxy server VM did not release this VMDK.

Resolution

• Use the command <lsof | grep -i namespace_UUID>.
  • This is the namespace UUID which contains the files to be deleted.  
• Take the Cartel ID and run the command <ps -u | grep -i <cartelID>>
   
  • The "vmx-filtPoll" will show the VM name for the VM holding onto this file. 
• Review the disks attached to this VM using vCenter UI (Edit settings) and remove any VMDK's you are trying to delete. 
  • NOTE: It may be necessary to power-off this VM to detach the disks.

Additional Information

• Investigating virtual disk file locks on vSAN