Password Remediation on SDDC fails with Error "Failed to update the password. User is not allowed to update or rotate PSC credentials. Please login using an alternate 'ADMIN' account to perform this operation."

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• Password remediation for [email protected] on VCSA fails when attempting to remediate the password.
  
  
• /var/log/vmware/vcf/operationsmanager/operationsmanager.log (on SDDC Manager)

 [c.v.v.p.r.v.PasswordManagerController,http-nio-127.0.0.1-7300-exec-1] updateOrRotatePassword {"operationType":"REMEDIATE","elements":[{"resourceName":"#################################","resourceType":"PSC","credentials":[{"username":"[email protected]","password":"*****"}]}]}
 [vcf_om,#####################] [c.v.v.p.r.v.helper.vel.SpecValidator,http-nio-127.0.0.1-7300-exec-1] Received request to Validate operation type REMEDIATE
YYYY-MM-DDTHH:MM:SS INFO  [vcf_om,####################################] [c.v.v.p.r.v.PasswordAudit:http-nio-127.0.0.1-7300-exec-1] PasswordAudit:Received request for credentials operation=REMEDIATE from user [email protected]
YYYY-MM-DDTHH:MM:SS ERROR [vcf_om,#################] [c.v.v.s.s.SoAuthenticationService,http-nio-127.0.0.1-7300-exec-1] [email protected] cannot be used for PSC password operations as it is managed by SDDC Manager.
YYYY-MM-DDTHH:MM:SS DEBUG [vcf_om,############################## [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7300-exec-1] Processing localizable exception User is not allowed to update or rotate PSC credentials. Please login using an alternate ADMIN account to perform this operation.
YYYY-MM-DDTHH:MM:SS ERROR [vcf_om,#################################] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7300-exec-1] [####] PASSWORD_MANAGER_USER_NOT_ALLOWED_PSC User is not allowed to update or rotate PSC credentials. Please login using an alternate ADMIN account to perform this operation.
com.vmware.vcf.passwordmanager.exception.PasswordManagerException: User is not allowed to update or rotate PSC credentials. Please login using an alternate ADMIN account to perform this operation.
    at com.vmware.vcf.passwordmanager.service.SsoAuthenitcationService.validateWithExistingSOSUser(SsoAuthenticationService.java:164)
    at com.vmware.vcf.passwordmanager.rest.v1.PasswordManagerController$UpdateOrRotatePasswordManagerController.java:208)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.security.authorization.method.AuthorizationManagerBeforeMethodInterceptor.invoke(AuthorizationManagerBeforeMethodInterceptor.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)

Environment

VMware Cloud Foundation (VCF) 5.x
VMware Cloud Foundation 9.1

Cause

[email protected] is managed by SDDC Manager and cannot be used for PSC password operations.

Resolution

In order to rotate the password for "[email protected]", proceed to create an alternate administrator account on SDDC Manager.

Step 1: Create the user in vCenter SSO

• Log in to the management domain vSphere Client as [email protected].
• Go to Administration > Single Sign On > Users and Groups.
• On the Users tab, select the vsphere.local domain and click Add.
• Enter a username (Ex: Admin) and password and save. 

Step 2: Give the account admin access in SDDC Manager

• Log in to SDDC Manager as [email protected].
• Go to Administration > Single Sign On (or Users and Groups, depending on your build).
• Click + User or Group.
• Choose the domain (vsphere.local), search for Admin in the user or group created in Step 1, and select the role ADMIN.
• Click Add.
• Log out, then test by logging in as the new account(Admin) into SDDC Manager.

Step 3: Update the [email protected] password in SDDC Manager

• Log in to SDDC Manager with the new account(Admin).
• Go to Administration > Security > Password Management.
• Find the vCenter entry for [email protected] , click the three dots next to it and choose Update Password.
• Enter the new password, confirm it and submit.
• Open the Tasks panel at the bottom and wait until the task shows Successful.

     Validate:

• Log in to the vSphere Client as [email protected] with the new password to confirm it works.
• Refresh Password Management and check the account status is Active and the last-modified time has updated.
• Update the password anywhere else that uses this account, such as backup tools, monitoring, scripts and your password vault.

Additional Information

If the above steps do not resolve the issue, please navigate to SDDC Manager > Single Sign-On and assign the required Admin role to the newly created account in SDDC Manager GUI