vCenter Server does not Automatically Provision VCLs Virtual Machines(VMs)

Products

• VMware vCenter Server
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

Assist customers to solve this issue.

Symptoms:

The vCenter  Server does not automatically deploy vCLs after attempting retreat mode due to an agency in yellow status.

wcp.log remain in the deletion and destroying agent loop. There are no entries to create an agency.

YYYY-MM-DD debug wcp [dataprovider/cluster_agencies.go:96] [opID=vCLS] Successfully deleted from the database record (domain-c24137) 

YYYY-MM-DD info wcp [eamagency/retreat.go:58] [opID=vCLS] Successfully destroyed agency, retreat operation completed

 

There  is no relevant  information in eam.log

In the eam_api-x.log you may see an error similar to the following:   

YYYY-MM-DD |  INFO | vlsi | LocalizationFilter.java | 108 | API COMPLETE: Facade for ClusterAgent(ID: 'Agent:3828c184-2c7a-4e15-9e2f-6059e3eb17a1:null').queryRuntime[opId=668781274, sessionId=2A280743]. Result:
eam.Agent.RuntimeInfo {
   issue = <unset>,
   goalState = 'enabled',
   entity = 'Agent:3328c105-2c3a-4e15-9e2f-6059e3eb17a1:null',
   status = 'yellow',
   vmPowerState = poweredOff,
   receivingHeartBeat = false,
   vmIp = <unset>,
   esxAgentResourcePool = <unset>,
   esxAgentFolder = <unset>,
   installedBulletin = <unset>,
   vmHook = <unset>,
   agency = 'Agency:10458969-7b9e-4ac9-b14e-45f6e2uyt5nvvc:null',
   vm = <unset>,
   installedVibs = <unset>,
   vmName = '',
   host = <unset>,

 

Environment

VMware vCenter Server 8.0.x
VMware vCenter Server 7.0.x
VMware vCenter Server 7.0.0

Cause

There are EAM agencies in yellow status.

Resolution

1. Check if there are any agencies in yellow color.
2.  Restarting EAM usually fixes the problem.

service-control --stop vmware-eam && service-control --start vmware-eam

3. After restarting the services follow the steps 1 through 8 from https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.resmgmt.doc/GUID-F98C3C93-875D-4570-852B-37A38878CE0F.html  or How to Disable vCLS on a Cluster via Retreat Mode  to enable "Retreat Mode".
4. Wait for 1 to 2 minutes to make sure the VCLs VM's were removed from the cluster.
5. Disable retreat mode by changing the config.vcls.clusters.domain-c<number>.enabled  value to "true".

Additional Information

How to Disable vCLS on a Cluster via Retreat Mode

Impact/Risks:

High because the DRS does not properly work causing an imbalance of cluster resources.