Decommission vSAN Data Protection/snap service appliance and all its files from file explorer in vCenter.

Products

• VMware vSAN

Issue/Introduction

When attempting to power on a virtual machine in the vSphere Client, the following error may appear if a backup or copied virtual machine disk was manually deleted using File Explorer from the vSAN datastore folder.

"Cannot open the disk '<path-to-vmdk>' or one of the snapshot disks it depends on. The system cannot find the file specified."
"File /vmfs/volumes/vsan:527###########-e3d#############/b0d#####-e6##-10##-c3##-005#########/TESTG.vmdk was not found Failed to start the virtual machine. Module Disk power on failed. Cannot open the disk  /vmfs/volumes/vsan:527###########-e3d#############/b0d#####-e6##-10##-c3##-005#########/TESTG.vmdk' or one of the snapshot disks it depends on. The system cannot find the file specified File system specific implementation of GetFileAttriutes[file] failed File system specific implementation of GetFileAttributes[file] failed File system specific implementation ofGetFileAttributes[file] failed File system specific implementation of GetFileAttributes[file] failed File system specific implementation of GetFileAttributes[file] failed VMware ESX cannot find the virtual disk " /vmfs/volumes/vsan:527###########-e3d#############/b0d#####-e6##-10##-c3##-005#########/TESTG.vmdk". Verify the path is valid and try again. Completed"

Environment

VMware vSAN 8.x
VMware vSAN Data Protection
VMware vSAN Snapshot Service Appliance
VMware Live Recovery Appliance 9.0.x

Cause

• Removing a Backup/Copied Virtual Machine Disk from a vSAN Datastore Using File Explorer may lead to source VMDK deletion.

• When we manually clean up the protection group namespace, the namespace object itself does not get deleted as it has snapshots, but the files inside the namespace are deleted. vpxd triggers an unlink to the vmdk causing the running point of the vm to be deleted.

Resolution

Steps to decommission and cleanup Snapservice / vSAN Data protection.

1. Power OFF appliance VM.

2. Run the attached script(cleanupSs.py) to disable snap service for a cluster.

• The script does the following:

• • Unregister snap service extension if present.
  • Disable snap service on the specified cluster
  • The script must be copied into the VC that requires cleanup and execute the following command:
    # python cleanupSs.py --vch <VC IP> --cls <cluster moref> --user <administrator user>

3. Unregister snapservice extensions from vCenter MOB

• • com.vmware.snapshotservice
  • com.vmware.vsan.snapshot.manager.client
  • com.vmware.vcAps

4. Delete snapstore.db

• Take master node from vSAN cluster.
• ssh to master node.
• Navigate to folder cd /vmfs/volumes/vsanDatastore/snapstoreDB/
• Mandatory: Keep a local copy of this db in case you need to perform recovery of Protection Groups and its snapshots later
• delete snapstore.db

Additional Information

If using Live Site Recovery 9.0.3 and above you can reconfigure from the VAMI interface to reconfigure the snapshot service for continued usage.