Clarification of Group membership Issue in Service Account Post vCenter 8.0U2 Upgrade in ELM Configuration.

Products

• VMware vCenter Server
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

This article exists to guide users facing this issue, helping them to perform vCenter operations successfully by ensuring the service account is in the correct group.

Additional symptoms: Post vCenter Upgrade to version 8.0U2 in an ELM setup with VC 7.x partners, service accounts might lose access to certain groups if there are changes made to the group membership list compared to last release. One such issue occurred for SPS service after they updated their group membership list in the vCenter 8.0U2 release.

Look for specific error messages or other unique details in the logs:
Service accounts may show errors about missing permissions or roles. An example is the SPS service account, which showed errors related to group membership in the following log:

Log File: /storage/log/vmware/vmware-sps/sps.log

The main issue is the SPS service account being absent from the Administrators group, resulting in errors and failed operations as shown in the log entries below:

yyyy-mm-ddThh:mm:ss.mssZ [main] ERROR ... - Failed to retrieve service content
yyyy-mm-ddThh:mm:ss.mssZ [main] ERROR ... - Caught exception - VpxdException: Error occurred while retrieving service content

Cause

This problem may arise in an ELM setup when one of the vCenter is upgraded to version 8.0U2 (which contains the fix for this issue), while other linked vCenter servers remain on version 7.x (which does not yet contain the fix).

Resolution

This issue has been resolved in vCenter Server 7.0 U3q. To download go to Broadcom Support Portal

To resolve the issue, follow any one the below options.

Option 1:

To address this issue temporarily, re-add the respective service account to the necessary group. Using the SPS service account as an example, follow these steps:

1. Take offline snapshots of all vCenters in the ELM setup.
2. Access the vCenter in question via SSH with root privileges.

3. Run the following commands to check if SPS user is in all of them: 

  /usr/lib/vmware-vmafd/bin/dir-cli group list --name Administrators
  /usr/lib/vmware-vmafd/bin/dir-cli group list --name ServiceProviderUsers

    /usr/lib/vmware-vmafd/bin/dir-cli group list --name ActAsUsers

     

     For example, while running the: /usr/lib/vmware-vmafd/bin/dir-cli group list --name ServiceProviderUsers 

     You will get an output similar to: CN=sps-xx-xx-xx-xx-xx,cn=xyz,dc=vcenter,dc=xyz, Note the "sps-xx-xx-xx-xx-xx" section.

     The SPS user must be missing in the Administrators section.

 

4. Run the following commands to add the SPS service account to the Administrators group:

  /usr/lib/vmware-vmafd/bin/dir-cli group modify --name Administrators --add sps-xx-xx-xx-xx-xx

 

5. Restart the SPS services using: service-control --stop sps && service-control --start sps

 

Option 2:

1. Remove the cache of the sps user account: rm /var/cache/svcaccounts/sps/.sps
2. Restart the sps service:  vmon-cli --restart sps