Snapshot scheduled task fails due to missing group permissions

Products

• VMware vSphere ESXi

Issue/Introduction

This article outlines troubleshooting steps for snapshot scheduled tasks which includes taking a snapshot and deleting.

Symptoms: 

• Scheduling snapshot fails with generic error "Failed to get persistable Token" on vCenter.
• Snapshot schedule tasks can be performed on 8.0.U3 as we do not have this feature in previous versions.

 

You may observe similar errors in the vpxd log:

2024-08-09T08:02:57.744Z info vpxd[06875] [Originator@6876 sub=SsoClient opID=lxuwiyzq-184406-auto-3yag-h5:70051570-30] Successfully acquired token:SamlToken [subject={Name: vpxd-########-####-####-####-###########; Domain:vsphere.local},groups= [{Name:Users; Domain:vsphere.local}, {Name: SolutionUsers; Domain:vsphere.local}, {Name:SystemConfiguration.Administrators; Domain:vsphere.local}{Name:ComponentManager.Administrators; Domain:vsphere.local}, {Name:LicenseService.Administrators;Domain:vsphere.local},{Name: Everyone;Domain:vsphere.local}],delegationChain=[], startTime=2024-08-09 08:02:57.688,endTime=2024-08-0916:02:57.688,renewCount=0,delegableCount=10,isSolution=true, type=Saml_HOK]
2024-08-09T08:02:57.798Z error vpxd[06875] [Originator@6876 sub=MoScheduledTask opID=lxuwiyzq-184406-auto-3yag-h5:70051570-30]Failed to get persistable Token: Unexpected SOAP fault: ns0:InvalidRequest; request failed.

 

So, when comparing the roles we see ActAsUsers group is missing:

2024-08-09T08:02:57.744Z info vpxd[06875] [Originator@6876 sub=SsoClient opID=lxuwiyzq-184406-auto-3yagh5:70051570-30] Successfully acquired token: SamlToken [subject={Name:vpxd-########-####-####-####-###########; Domain:vsphere.local}, groups=[{
Name: Users; Domain:vsphere.local},
Name: SolutionUsers; Domain:vsphere.local},
Name: SystemConfiguration.Administrators; Domain:vsphere.local},
Name: ComponentManager.Administrators; Domain:vsphere.local},
Name: LicenseService.Administrators; Domain:vsphere.local},
Name: Everyone; Domain:vsphere.local}]

 

Below role needs to be added:

Name: ActAsUsers; Domain:vsphere.local

 

Once added you may see below entries:

2024-08-12T22:21:13.755Z info vpxd[06985] [Originator@6876 sub=SsoClient] Successfully acquired token: SamlToken [subject= {Name: vpxd-########-####-####-####-###########; Domain:vsphere.local}, groups=[{

Name: Users; Domain:vsphere.local}, {
Name: SolutionUsers; Domain:vsphere.local}, {
Name: SystemConfiguration.Administrators; Domain:vsphere.local}, {
Name: ActAsUsers; Domain:vsphere.local}, {
Name: ComponentManager.Administrators; Domain:vsphere.local}, {
Name: LicenseService.Administrators; Domain:vsphere.local}, {
Name: Everyone; Domain:vsphere.local}]

 

Environment

VMware vSphere ESXi 8.0.3

Cause

This issue might occur when there are missing group permissions which is responsible to initiate the scheduled snapshot task on vCenter.

Resolution

Run the below command on vCenter cli to modify the permissions followed by relog in.

/usr/lib/vmware-vmafd/bin/dir-cli group modify --name ActAsUsers --add vpxd-########-####-####-####-###########

Set this permission for modifying a user/group for automated tasks that have to authenticate to vCenter Server, and to ensure that the tasks do not stop running.

 

Additional Information

ActAsUser:

1.Creating a solution user "ActAsUser" makes the solution user a member of the built-in administrators or users group. In other words, determines whether the solution user has administrative privileges. This role enables users to act on behalf of other users.

3.Members of Act-As Users are allowed to get Act-As tokens from vCenter Single Sign-On.