vCLS virtual machines not created after delete entry in vpx_ext_data

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• After deleted entry in the table vpx_ext_data of vCenter database, all vCLS VMs were removed automatically. 
• But vCLS VMs were not re-created any more.
• Disable retreat mode not help. 
• Restart vCenter services not help.
• In wcpsvc.log, there are messages liking this:

YYYY-MM-DDTHH:MM:SS.723Z error wcp [eamagency/create.go:72] [opID=vCLS] Unable to create entity in db for cluster agency: ERROR: duplicate key value violates unique constraint "hdcs_cluster_agencies_pkey" (SQLSTATE 23505)

Environment

VMware vCenter Server 7.0.x

Resolution

To delete entry in table vpx_ext_data will automatically vCLS agency VMs from vCenter. However the corresponding agency in table hdcs_cluster_agencies_pkey did not clean at the same time. That caused WCP can not create a new entry of vCLS.

To resolve the issue, need to manually remove the agency of cluster in the table hdcs_cluster_agencies_pkey too.
1. Ensure the Retreat Mode has been disabled for the cluster with the KB How to Disable vCLS on a Cluster via Retreat Mode (91890) . That is config.vcls.clusters.domain-c<number>.enabled=True
2. SSH to vCenter Server
2. Access the vPostgres database with the KB Interacting with the vCenter Server Appliance 6.5/6.7/7.0 embedded vPostgres Database (2147285)
3. Locate the cluster agency entry for cluster where vCLS VMs not created. 
VCDB=# SELECT cluster_moref, agency_moref FROM hdcs.hdcs_cluster_agencies WHERE cluster_moref='<domain-number-of-cluster>';

For example:
VCDB=# select cluster_moref, agency_moref FROM hdcs.hdcs_cluster_agencies WHERE cluster_moref='domain-c30019';
 cluster_moref |             agency_moref
---------------+--------------------------------------
 domain-c30019 | 7f4ec319-053d-4dff-8dda-a69ee368a86e
(1 row)

4. Delete the cluster agency entry for the cluster:
VCDB=# delete from hdcs.hdcs_cluster_agencies where cluster_moref='<domain-number-of-cluster>';

The <domain-number-of-cluster> is same with step 3.

Once the agency ware deleted from hdcs.hdcs_cluster_agencies, the vCLS VMs should be immediately re-created.