vCLS virtual machines are inaccessible due to loss of storage connectivity within a vCenter server

Products

• VMware vSphere ESXi

Issue/Introduction

• Multiple vCLS virtual machines are inaccessible in vCenter UI due to the unavailability of their backing datastore.
• Deployment of new vCLS virtual machines by the ESX Agent Manager (EAM) fails due to residual references of the inaccessible vCLS persisting within the vCenter Server inventory.
• This issue does not affect environments running vSphere 8.0 Update 3 or later, as these versions introduce Embedded vCLS, which eliminates the need for a backing datastore.

Environment

vCenter Server 7.x

vCenter Server 8 Update 2 and earlier

Cause

This behavior is caused by the inaccessibility of the underlying datastores where the vCLS virtual machines reside.

Resolution

1. Enable Retreat Mode on the cluster containing the affected vCLS virtual machines. For detailed instructions, refer to Broadcom KB: Disable vCLS on a cluster via Retreat Mode.
   
   
2. If Retreat Mode fails to automatically clean up any of the vCLS virtual machines, log in to the ESXi Host Client where those specific vCLS reside and manually unregister them from the host inventory.
   
   
3. If the stale vCLS virtual machines have been removed from the ESXi hosts but still appear in the vCenter Server UI, purge the stale entries from the vCenter Server PostgreSQL database. For detailed instructions, refer to Broadcom KB: Remove stale VMs from postgres database.
   
   
4. Once all the stale vCLS entries are removed, disable Retreat Mode on the cluster. This will allow the ESX Agent Manager (EAM) service to successfully deploy new vCLS virtual machines.