After verify job some devices show synchronized but manual verify fails.

Products

• CA Privileged Access Manager (PAM)

Issue/Introduction

A Verify job was run against a target group to find out of sync target accounts, but it failed to mark multiple accounts as Unverified, even though they belonged to target devices in the group that were known be be offline/decommissioned. Subsequent manual verifications failed as expected.

Cause

The problem was caused by "orphaned" target accounts and applications, for which the corresponding target server entry had been marked as deleted in the PAM database already, but not the accounts and target application associated with it. This didn't cause errors in the PAM UI, but affected scheduled jobs. Once the scheduled job gets to an orphaned account, it will run into an error and stop, failing to process the remaining accounts in the job. By the time the problem was noticed, the logs of interest had rolled over already and we could not determine root cause. It did occur before the upgrade to 4.1.1.

There is no useful message in the tomcat log, but the message at the end of the job shows that the number of accounts processed is less than what would be expected. In the following message, the scheduled job ends with an error (result=false) and the number of accounts (608) is much smaller than it should have been. This job ran on a target group that contained about 1200 accounts.

2022-11-21T17:59:12.161+0000 WARNING [TestScheduler_Worker-24] com.cloakware.cspm.server.app.impl.VerifyAccountPasswordCmd.invoke VerifyAccountPasswordCmd.invoke, end: result=false, accounts=608, duration=732137.6ms

Resolution

If you observe such a problem in a current PAM release, collect the session and system logs (logs.bin) from your standalone PAM server, or in a cluster from all primary site cluster nodes, for review by PAM Support. The session logs are downloaded from the Sessions > Logs page. The system logs are downloaded from the Configuration > Diagnostics > Diagnostic Logs > Download page by clicking on the Download button to the right of the "Download System Diagnostics" label. Make sure to click only once, even though it may take a while before you see the logs.bin download starting.

Attach the logs to a new Support case and provide the job name as well as the time it ran. Also provide the number of target accounts in the target group that the job is defined with. This information can be obtained by editing the target group from the Credentials > Manage Targets > Target Groups page, clicking the Show button on the bottom right and then selecting the Target Accounts tab. The number of entries in the target accounts list will be seen in the bottom left, if the list is longer than one page.