vCLS VMs failing to power on in VC 7.x and VC 8.x due to Privilege check failed for user VSPHERE.LOCAL\vpxd-extension-xxxx

Products

• VMware vSphere ESXi

Issue/Introduction

• the cluster summary page for one or more clusters in vSphere Client shows the following error message:

vSphere DRS funtionality was impacted due to unhealthy state vSphere Cluster services caused by the unavailability of vSphere Cluster Service VMs.
vSphere Cluster Service VMs are required to maintain the health of vSphere DRS.

• One or more vCLS VM(s) have been deployed in the cluster, but they are not powered on.
• The events tab for the VCLs VM(s) contain warning events with the following message:

Privilege check failed for user VSPHERE.LOCAL\vpxd-extension-xxxx for missing permission.

 

• Trying to put a host int maintenance mode stops at a low percentage and no VMs are automatically migrated
• The VCLS VM's are deployed to the "EAM Agents" foder instead of the "VCLS" folder

Environment

VMware vSphere 7.0.x

VMware vSphere 8.0x

Cause

The reason for this is due to over privileging of the VPXD and VPXD-extension solution users accounts.

Resolution

Note: Please ensure to take a fresh backup or snapshot of the vCenter Server Appliance, before going through the steps below. In case the affected vCenter Server Appliance is a member of an Enhanced Linked Mode replication group, please be aware that fresh offline snapshots (in powered off state) or backups of all members of the replication group are required.

To fix the issue in the environment, please apply the following steps:

1. Download the fixAdministratorsGroup script attached to this KB.
2. Upload the script into the /tmp/ directory of the vCenter.
3. SSH into the vCenter using the root credentials.
4. Change into the /tmp directory

   # cd /tmp

5. Set the script as executable:

   # chmod u+x fixAdministratorsGroup

6. Run the following command to check for the users that need to be removed from the Administrators group:

   # python fixAdministratorsGroup.py -u [email protected] -p '<admin-pwd>' --action=check

7. Run the following command to fix the issue:

   # python fixAdministratorsGroup.py -u [email protected] -p '<admin-pwd>' --action=fix

8. Check in vSphere Client, if the vCLS VMs have been powered on successfully.

Additional Information

Without sufficient vCLS VMs in running state, DRS won't work.