Unable to power on VMs from vCenter with error "All required agent virtual machines are not currently deployed on host 'hostname''"

Products

• VMware vCenter Server

Issue/Introduction

• Unable to power on virtual machines from vCenter
   
• You see an error similar to: All required agent virtual machines are not currently deployed on host hostname.

/var/log/vmware/vpxd.log on vCenter::
XX:XX:XXTXX:XX:XX info vpxd[09951] [Originator@6876 sub=vpxLro opID=m8qy7sos-152286-auto-39i7-h5:70030902-f7-01] [VpxLRO] -- BEGIN task-150241 -- VMName -- Datacenter.ExecuteVmPowerOnLRO --
XX:XX:XXTXX:XX:XX info vpxd[09951] [Originator@6876 sub=cdrsPlmt opID=m8qy7sos-152286-auto-39i7-h5:70030902-f7-01] Vm [vim.VirtualMachine:vm-xxx,VMName] failed constraint check true on host [vim.HostSystem:host-xxxxx,esxifqdn] with <obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xml
ns="urn:vim25" versionId="7.0.3.0" xsi:type="LocalizedMethodFault"><fault xsi:type="InsufficientAgentVmsDeployed"><hostName>esxifqdn</hostName><requiredNumAgentVms>4</required
XX:XX:XXTXX:XX:XX info vpxd[09951] [Originator@6876 sub=cdrsPlmt opID=m8qy7sos-152286-auto-39i7-h5:70030902-f7-01] Dump section 1205596710441 created.
XX:XX:XXTXX:XX:XX error vpxd[09951] [Originator@6876 sub=vpxLro opID=m8qy7sos-152286-auto-39i7-h5:70030902-f7-01] [VpxLRO] Unexpected Exception: N3Vim5Fault28InsufficientAgentVmsDeployed9ExceptionE(Fault cause: vim.fault.InsufficientAgentVmsDeployed
--> )
--> [context]zKq7AVECAQAAAOIfcwEYdnB4ZAAAAto3bGlidm1hY29yZS5zbwAAmXksABdtLQAf6jKB88IHAWxpYnZpbS10eXBlcy5zbwCBndMHAYELfAgBAR+x6wKXHXN2cHhkAIIdYkABAj0N+wLG0vsCtdP7AqBmlIL1BGkBgjYGaQGCZBVpAYLHQmgBgpLsaAEA50kjAHWfIwDAZTcDh38AbGlicHRocmVhZC5zby4wAAS/Ng9saWJjLnNvLjYA[/context]
XX:XX:XXTXX:XX:XX info vpxd[09951] [Originator@6876 sub=vpxLro opID=m8qy7sos-152286-auto-39i7-h5:70030902-f7-01] [VpxLRO] -- FINISH task-150241
XX:XX:XXTXX:XX:XX info vpxd[09951] [Originator@6876 sub=Default opID=m8qy7sos-152286-auto-39i7-h5:70030902-f7-01] [VpxLRO] -- ERROR task-150241 -- VMName -- Datacente
r.ExecuteVmPowerOnLRO: vim.fault.InsufficientAgentVmsDeployed:
--> Result:
--> (vim.fault.InsufficientAgentVmsDeployed) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    hostName = "esxiname",
-->    requiredNumAgentVms = 4,
-->    currentNumAgentVms = 2
-->    msg = ""
--> }
--> Args:

Environment

vCenter 7.x

Cause

The error "All required agent virtual machines are not currently deployed on host 'hostname'" occurs because vCenter Server’s ESX Agent Manager (EAM) detects fewer ESX Agent Virtual Machines deployed on the host than required. The VPXD logs show that the host needs 4 ESX Agent Virtual Machines but only 2 are deployed, causing vCenter to block VM power-on operations due to this configuration mismatch.

Resolution

Action Plan:

Take a Snapshot of the vCenter before making any changes

1) Open: {vc-fqdn}/mob/?moid=EsxAgentConfigManager&vmodl=1
2) Open: updateComputeResourceAgentInfo
3) Paste this in the value VALUE field:
<computeResourceAgentInfo>
   <computeResource type="ComputeResource">CLUSTER MOID</computeResource>
   <numRequiredAgents>0</numRequiredAgents>
</computeResourceAgentInfo>


NOTE: Replace CLUSTER MOID with the ID of the problematic cluster (e.g. domain-c123).