Citrix MCS: Orphaned VMDK files remain on vSphere datastore after catalog deletion

Products

• VMware vSphere ESXi
• VMware Cloud Foundation

Issue/Introduction

When cleaning up a Citrix Machine Creation Services (MCS) environment by deleting a Machine Catalog, some `.vmdk` files or entire VM folders remain on the vSphere datastore despite the deletion task reporting success in vCenter.

 

• VMDK files are left behind in the datastore after the associated VM has been removed from Citrix MCS
• Multiple folders may exist for the same virtual machine name.
• Deleting a remnant vmdk file for a virtual machine that is no longer used fails with error: "Device or resource busy."
• Multiple folders may remain for the same VM name on the datastore.
• Tasks in vCenter show as successful, but files persist physically on storage."

Environment

• VMware vSphere ESXi
• VMware vCenter Server
• VMware Cloud Foundation (VCF)
• Citrix MCS on vSphere

Cause

• This issue stems from the 3rd party Citrix MCS process failing to issue or complete all necessary file deletion commands thorugh the vSphere API, or a process (such as a backup agent or the Citrix controller) maintaining a lock on the file during the deletion attempt.

 

Resolution

To resolve this issue and clean up the orphaned files, follow these steps:

1. Verify Usage:

• Ensure the orphaned VMDKs are not currently attached to any active Virtual Machines in the inventory.

2. Manual Deletion: 

• Navigate to the Datastore in the Datastore Browser. 
  • In the vSphere Client Navigate to Inventory -> Datastores & Clusters Tab -> Right-click datastore name -> Browse Files
    • See also TechDocs: Using Datastore Browser in vSphere Environment
  • Locate the orphaned folders or files and attempt to delete them manually.
  • Alternatively, use the ESXi Command Line Interface (CLI) commands to remove the files if you prefer granular control.

3. Handle Locked Files: If a "Device or resource busy" error occurs, identify the host holding the lock and ensure no 3rd party agents (e.g., backup software) are accessing the file.

• See related KBs:
  • Investigating Virtual Machine file locks on ESXi Host(s)
  • VM folder fails to delete from datastore with error "Cannot delete file"

 

4. Citrix Coordination: Consult Citrix documentation or support to verify if there are specific dependencies or if API calls are being properly issued from the Citrix Delivery Controller.

Additional Information

• See also Citrix Documentation:  Manage machine catalogs