Misleading PAM-CM-0762 Authentication failed error when a service account fails to update another Active Directory target account

Products

CA Privileged Access Manager (PAM)

Issue/Introduction

We had temporary failures getting AD accounts updated using a service account that is configured with rights to update the passwords of the managed accounts, as well as unlock them. There is no useful information in the tomcat log at any log level. The update attempt fails with an authentication error, code 1604. The service account itself has a valid password and authenticated successfully, but could not update the password of the managed account because of a problem on the AD side. The tomcat logs at Info log level did not show any error from the service account, only the failed authentication attempts by the managed account.
Here is an example of two consecutive log lines, the first one reporting successful authentication by the service account, the next one already belonging to the next login attempt by the managed account:
...
May 29, 2020 3:19:05 AM com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager loginToActiveDirectoryServer
INFO: Successfully authenticated to Active Directory and set the last known good host to 'xxxx.mydomain.net'
May 29, 2020 3:19:05 AM com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager updatePasswordInActiveDirectory
INFO: Retrieve the DN from the Active Directory for user, changemypassword,CN=Changemy Password,CN=Users,DC=mydomain,DC=net
...
Between these two log lines the service account ran into an error making a call into AD, but this error is not seen in the tomcat logs nor shown on the UI, hindering determination of root cause and resolution of the problem.

Environment

Release : Any PAM release as of March 2022

Component : PRIVILEGED ACCESS MANAGEMENT

Cause

The workflow for a password update for an Active Directory target account (managed account) by a service account is as follows:

1) PAM tries to logon to AD using the DN of the managed account and the new password - This will fail with LDAP error 49 (invalid credentials)

2) PAM tries to logon to AD using the UPN of the managed account and the new password - This will fail with LDAP error 49

3) PAM logs on as the service account, unlocks the managed account if needed and sets the new password.

4) PAM tries to logon to AD using the DN of the managed account and the new password - This will be successful, if 3) was successful, but fail otherwise with LDAP error 49

5) If 4) failed, PAM tries to logon to AD using the UPN of the managed account and the new password - This will fail with LDAP error 49

Steps 1 and 2 are useful for the case where a PAM admin enters a password on the UI, because that typically is the correct current password. However, this is done for any type of password update, even when a scheduled job generates a new password, in which case those attempts are bound to fail. This is the current workflow in PAM that may change in future releases.

The main flaw with regard to the problem discussed here is that any error during step 3 is caught without logging or otherwise saving the error. PAM just proceeds to step 4 This is not always wrong as it is possible that an error is returned, such as a timeout error, even though the password of the managed account in fact was or is being changed successfully. But not giving the PAM admin any information on what error was encountered obscures the problem and delays resolution. Since PAM just continues with steps 4 an 5, the last error encountered will be an authentication error and that is what will be reported on the UI. In the Account Passwords Update Attempts report the failed password update will show error code 1604.

Resolution

For now PAM Engineering is not changing the program flow, but since PAM 3.4.2 there is additional logging in the tomcat log that should allow you to see the LDAP error that the password update attempt ran into. Here is an example:

Mar 01, 2022 3:18:30 PM com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager loginToActiveDirectoryServer
INFO: Successfully authenticated to Active Directory and set the last known good host to 'x.x.x.x'
Mar 01, 2022 3:18:30 PM com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager getDistinguishedName
INFO: Derived the Distinguished Name (DN) 'xxxx' from the Target Account having ID 'dddd'
Mar 01, 2022 3:18:30 PM com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager updatePasswordInActiveDirectory
SEVERE: Failed with Naming Exception
javax.naming.directory.InvalidAttributeValueException: [LDAP: error code 19 - 0000052D: AtrErr: DSID-0319102B, #1:
0: 0000052D: DSID-0319102B, problem 1005 (CONSTRAINT_ATT_TYPE), data 0, Att 9005a (unicodePwd)
]; remaining name 'CN=...'
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3274)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3207)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2998)
at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1503)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:277)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:192)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:181)
at javax.naming.directory.InitialDirContext.modifyAttributes(InitialDirContext.java:167)
at com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager.updatePasswordInActiveDirectory(WindowsDomainServiceTargetManager.java:642)

...

Mar 01, 2022 3:18:30 PM com.cloakware.cspm.server.plugin.targetmanager.WindowsDomainServiceTargetManager loginToActiveDirectoryServer
INFO: Attempting authentication to Active Directory server 'x.x.x.x'

In the example above the password update attempt ran into a "CONSTRAINT_ATT_TYPE" error caused by a minimum password age constraint on the Active Directory side. All updates would fail with this error, if the minimum password age had not been reached yet since the last successful update.

As discussed in the Cause section above, this error occurred in step 3, and there were two "LDAP: error code 49" errors before (from steps 1 and 2), and two after (steps 4 and 5) the above constraint error.

Removing the minimum password age constraint resolved the problem.