Unable to delete VM folder from vSAN datastore

Products

• VMware vSAN

Issue/Introduction

VM has already been deleted, but the folder remains on the vSAN datastore, and appears empty.

Environment

vSAN 8.x, 9.x

Cause

There are hidden files in the VM folder/namespace, that are preventing deletion of the folder.

 

Resolution

 

If unsure about any of these steps, please open a case with Broadcom Support.

 

 

From an SSH session on an ESXi host in the cluster, cd to the vSAN namespace, and run ls -l to show hidden files.

cd /vmfs/volumes/vsanDatastore/"VMname"/

ls -l

 

Look for files with a .lck extension.

If these files have a vSAN UUID in the name, check what that object refers to.

- vSAN UUIDs have the format ########-####-####-####-############

/usr/lib/vmware/osfs/bin/objtool getAttr -u "object UUID"

 

Look at the object path at the bottom of the output.

Object path: /vmfs/volumes/vsan:####.../########-####-####-####-############/"name.vmdk"

 

Check if that file, at the exact path called out, actually exists on the vSAN datastore.

If it does not, then this is a stale lock

 

Perform an owner abdication on the object UUID:

 

1. Find the current DOM owner (host UUID) from the output.

cmmds-tool find -t DOM_OBJECT -f json -u "Object UUID" | head

 

2. Map the owner UUID from the above output to an ESXi host.

cmmds-tool find -t HOSTNAME -f json -u "Owner UUID" | grep content

 

3. Log onto an SSH session on the host identified in the above output.

 

4. Run the following command to force the object to abdicate ownership:

vsish -e set /vmkModules/vsan/dom/ownerAbdicate "Object UUID"

 

5. Confirm that the owning ESXi host has changed by rerunning the command from Step 1.

 

If abdication has succeeded, the .lck file will be removed.

 

Repeat for all .lck files in the VM namespace.

 

When all .lck files have been cleaned up, deletion of the namespace folder will be successful.