Error: vSphere DRS functionality was impacted due to unhealthy state vSphere Cluster services caused by the unavailability of vSphere Cluster Service VMs.
search cancel

Error: vSphere DRS functionality was impacted due to unhealthy state vSphere Cluster services caused by the unavailability of vSphere Cluster Service VMs.

book

Article ID: 385568

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • Powered off vCLS VMs fails to delete even after enabling retreat mode on the affected cluster.
  • In vCenter Server you can find the below log snippets in /var/log/vmware/wcp/wcpsvc.log 
    YYYY-MM-DDTHH:MM:SS.347Z error wcp [eamagency/retreat.go:49] [opID=vCLS] Unable to destroy agency (Agency:########-####-####-####-###########): ServerFaultCode: "com.vmware.eam.DisabledOperationException: Invocation of disabled API Agency.DestroyAgency"

Environment

VMware vCenter 7.x
VMware vCenter 8.x

Cause

vCLS ESX Extension is in Warning state and therefore unable to call destroy agency vCLS API. 

Resolution

1. Enable Retreat Mode.

Enable Retreat mode via vCenter GUI.
Cluster > Configure > General > Edit vCLS Mode > Retreat Mode. 

2. Re-enable the API by leveraging the EAM MOB link here:

https://<vCenter_FQDN_or_IP_address>/eam/mob?vmodl=1

Under Methods section look for com.vmware.vim.binding.vmodl.ManagedObjectReference and click enable against it.

In the new window, modify the contents to include the "MOID" of the target cluster:

By default: <cluster type="ClusterComputeResource" serverGuid="Server GUID">MOID</cluster> 
Our modification: <cluster type="ClusterComputeResource" serverGuid="Server GUID">domain-cx</cluster> 

Click Invoke Method.

The powered off vCLS VMs were deleted automatically after the above change.

3. Disable Retreat mode via vCenter GUI.

Cluster > Configure > General > Edit vCLS Mode > System Managed.

# sed '/<vcls>/,/<\/vcls>/d' -i /etc/vmware-vpx/vpxd.cfg

Refer to Disable vCLS on a Cluster via Retreat Mode article For Versions Prior to vSphere 7.0 U3o and 8.0 U2, Using the vSphere Client for more information. Remove the Retreat Mode configurations from the vpxd.cfg file using the above command.

New vCLS VMs were deployed and powered on successfully.

Additional Information