How to retrieve password from Secondary Site when the Primary Site is down

Products

CA Privileged Access Manager (PAM) CA Privileged Access Manager - Cloakware Password Authority (PA) CA Privileged Access Manager - Server Control (PAMSC)

Issue/Introduction

There are times when the Primary Site can become unavailable.
This article demonstrates how the Secondary Sites can retrieve password from local DB in such situation without turning off the cluster and unlocking DB.

Environment

Release : 3.3.0, 3.3.1

Component : PRIVILEGED ACCESS MANAGEMENT

Cause

Secondary Site requires Primary Site to be available to service the credentials.
But when the Primary Site is unavailable, Secondary Sites would not provide credentials and require turning off cluster and unlocking DB.

Resolution

This is based on PAM 3.3.0 and 3.3.1
In short, whether you can retrieve password from Secondary Site node's local DB is based on the "Run Secondary Site in Operationally Safe Mode [x]" under "Disaster Recovery" tab.

Following Demonstration is based on the following setup.
[Primary Site]
pam330a1.ldap1.lab
pam330a2.ldap1.lab
pam330a3.ldap1.lab

[Secondary Site]
pam330a4.ldap1.lab
pam330a5.ldap1.lab

Primary Site is disconnected from network for this testing to make it unavailable.

Use case: In the Cluster setting at the Primary Site, select "Security Safe" mode.

At the Secondary Site, "Disaster Recovery" tab appears. When "Security Safe" mode was chosen at the Primary Site, the Secondary Site nodes would have "Run Secondary Site in Operationally Safe Mode" unchecked.

At pam330a4.ldap1.lab Disaster Recovery tab, select "Operationally Safe" mode

At pam330a5.ldap1.lab Disaster Recovery tab, select "Security Safe" mode.

Check the "Run Secondary Site in Operationally Safe Mode" for pam330a4.ldap1.lab

Leave the setting as is on pam330a5.ldap1.lab

Disconnect the Primary Site

Confirm the Replication Status at the Secondary Site nodes appear as Timed out

Test if Secondary Site can retrieve password.

OUTCOME

Pam330a4.ldap1.lab can retrieve password and access to target application continues to work.

Pam330a5.ldap1.lab fails to retrieve password with error.

Use case 2: In the Cluster setting at the Primary Site, select "Operationally Safe" mode (default).

At pam330a4.ldap1.lab, it takes the default setting for "Operationally Safe" mode so "Run Secondary Site in Operationally Safe Mode" option is already checked.

At pam330a5.ldap1.lab, "Run Secondary Site in Operationally Safe Mode" is also checked by default inheriting the setting at the Primary Site.

Now this will be unchecked for testing.

Disconnect the Primary Site

Confirm the Replication Status at the Secondary Site nodes appear as Timed out

Test if Secondary Site can retrieve password.

OUTCOME

Pam330a4.ldap1.lab continues to retrieve password.

Pam330a5.ldap1.lab launches the app but as it fails to retrieve credentials it closes by itself.

Sessions log shows the Primary Site is not available and an error is reported.

Change to "Run Secondary Site in Operationally Safe Mode" on pam330a5.ldap1.lab.

Then the credential can be retrieved.

Additional Information

[PAM 3.3.0]
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/layer7-privileged-access-management/privileged-access-manager/3-3/deploying/set-up-a-cluster/configure-a-cluster.html

[PAM 3.3.1]
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/layer7-privileged-access-management/privileged-access-manager/3-3-1/deploying/set-up-a-cluster/configure-a-cluster.html

6. On Secondary sites, the Disaster Recovery tab defines the behavior of an individual secondary site member in case the primary site fails. For a secondary member to behave in Operationally Safe mode, keep the Run Secondary Site in Operationally Safe Mode checkbox selected. To run in Security Safe mode, clear this checkbox.