vCLS VMs do not deploy due to the EAM Agency showing as "Disabled" following a vSAN Cluster Shutdown
Article ID: 344893
Updated On:
Products
Issue/Introduction
- vCLS VMs will not deploy in a cluster, even after enabling and disabling retreat mode
- Messages similar to the following are seen in /var/log/vmware/vpxd/vpxd.log
--> Result:
--> (vim.fault.InvalidName) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>,
--> name = "config.vcls.clusters.",
--> entity = <unset>
--> msg = ""
--> }
--> Args:
-->
--> Arg name:
--> "config.vcls.clusters."
- Recent logs from EAM show that the agency is disabled. /var/log/vmware/eam/eam.log
Updating issues:
New issues:
[
eam.issue.AgencyDisabled {
description = <unset>,
time = YYYY-MM-DD HH:MM:SS,
key = 0,
agency = 'Agency:#######-#####-#####-#####-###########:null',
solutionId =
'VSPHERE.LOCAL\vpxd-extension-########-####-####-####-################',
agencyName = 'vCLS',
solutionName = ' ',
}
]
Environment
Cause
Resolution
In scenarios where EAM agencies are not functioning as expected following a vSAN shutdown, the following workaround should be applied.
For all future operations, it is strongly recommended to use the Restart Cluster Wizard after completing the vSAN Shutdown Wizard to ensure proper reinitialization of services.
Prerequisite:
Ensure that all virtual machines (VMs) running on the vSAN cluster are accessible and operating normally before proceeding.
Workaround Steps
- Retrieve the Cluster MOID
- Open the vSphere Client and navigate to the vSphere cluster.
- Check the browser's address bar, where the cluster's MOID (Managed Object ID) will be shown in format of domain-c####. (e.g. domain-c10; do not copy anything after the ":")
- Retrieve the Server GUID (especially important when using vCenter Enhanced Linked Mode)
- Open the vSphere Client and navigate to the relevant vCenter of affected cluster.
- Check the URL in the browser's address bar. The "Server GUID" will be shown in format of Folder:group-d1:########-####-####-####-############. The needed GUID is the 32 hexadecimal-digits string. (Do not copy "Folder:group-d1:" as they are not part of the GUID)
- To access the EAM Managed Object Browser (MOB), open a browser and go to:
https://<VCENTER_IP>/eam/mob?vmodl=1 - To enable EAM Agencies for the Cluster, click on the link under "Methods" called "enable". A pop-up window will appear.
- In the value field, input the following XML, replacing "MOID" with the domain-c#### value and "Server GUID" with the actual server GUID:
<cluster type="ClusterComputeResource" serverGuid="Server GUID">MOID</cluster> - Click "Invoke Method".
- Wait for vCLS VMs to deploy.
- Allow a few minutes for the cluster to automatically deploy the vSphere Cluster Services (vCLS) VMs.
- A task named "Enable system VMs" will be triggered within the vCenter UI.
If the Task Fails
- Repeat Steps 1–5 above, and in Step 6 use the simplified XML (excluding serverGuid):
<cluster type="ClusterComputeResource">MOID</cluster> - Click Invoke Method.
- The "Enable system VMs" task should now complete successfully.
If vCLS VMs Are Still Not Deployed
Put the cluster into retreat mode, then disable retreat mode to force a refresh of the deployment process, steps are mentioned here: Disable vCLS on a Cluster via Retreat Mode
Additional Information
when EAM agents are disabled: Despite the Supervisor VMs being healthy and all pods running, the WCP service can throw errors such as
API call to VMware ESX Agent Manager (eam) failed. Details 'ServerFaultCode: "com.vmware.eam.DisabledOperationException: Invocation of disabled API Agent.markAsAvailable"
After following restarting the vSAN Cluster and Enabling the EAM Agencies for the cluster in EAM Managed Object Browser (MOB), you may need to restart the WCP service on vCenter.
vmon-cli -r wcp
Feedback
