Unable to deploy vCLS VMs by switching to Retreat mode

Products

• VMware vSphere ESXi
• VMware vCenter Server 8.0
• VMware vCenter Server 7.0
• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 8.0

Issue/Introduction

• DRS not able to migrate VMs.
• The existing vCLS VMs were in a powered off state and were not able to get power ON.
• Tried to put the cluster in retreat mode, but found the following error in the vCenter - /var/log/vmware/vpxd/vpxd.log
  

  YYYY-MM-DDTHH:MM:SS error vpxd[06977] [Originator@6876 sub=Default opID=wcp-vCLS-1c] [VdbStatement] Bind parameters:
  YYYY-MM-DDTHH:MM:SS error vpxd[06977] [Originator@6876 sub=Default opID=wcp-vCLS-1c] [VdbStatement] [0]datatype: 3, size: 8, arraySize: 1
  YYYY-MM-DDTHH:MM:SS error vpxd[06977] [Originator@6876 sub=Default opID=wcp-vCLS-1c] [VdbStatement] value[0] = ####
  YYYY-MM-DDTHH:MM:SS error vpxd[06977] [Originator@6876 sub=Default opID=wcp-vCLS-1c] [Vdb::IsRecoverableErrorCode] Unable to recover from 23505:1
  YYYY-MM-DDTHH:MM:SS error vpxd[06977] [Originator@6876 sub=Default opID=wcp-vCLS-1c] [VdbStatement] SQLError was thrown: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_auth_tenant_mgmt_rs"
  --> DETAIL: Key (entity_id)=(####) already exists.;
  --> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_AUTH_TENANT_MGMT_RS(ENTITY_ID) VALUES (?)"
  YYYY-MM-DDTHH:MM:SS warning vpxd[06977] [Originator@6876 sub=TenantManager opID=wcp-vCLS-1c] DB constraint. Trying to add a duplicated record: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_auth_tenant_mgmt_rs"
  --> DETAIL: Key (entity_id)=(####) already exists.;
  --> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_AUTH_TENANT_MGMT_RS(ENTITY_ID) VALUES (?)" 

Environment

• VMware vSphere ESXi 7.x
• VMware vSphere ESXi 8.x
• VMware vCenter Server 7.x
• VMware vCenter Server 8.x

Cause

Stale entry of vCLS VMs in the VCDB is causing trouble in the deployment of New vCLS VMs.

Resolution

The resolution requires removing stale entries present in vCenter Server's Database.  Contact Broadcom Support  for assistance.