PAM Server Refusing Connection Two-Node Cluster - Recovery and Best Practices

Products

• CA Privileged Access Manager (PAM)

Issue/Introduction

In a two-node CA PAM cluster, the secondary node refuses connections and displays the following error after a reboot:

[PAM] PAM-UI-1428: Bootstrap node for the primary site is unavailable.

This indicates the cluster has entered Quorum-Loss Mode, preventing normal operations and user logins.

Cause

A two-node cluster enters Quorum-Loss Mode when one node loses communication with the active primary node.

Resolution

Prerequisites

Before proceeding with recovery, ensure backend access is available in case Broadcom Support needs to assist:

1. Verify the ssh-debug patch is installed and the SSH debug service is running on any accessible node.

2. Note: If the ssh-debug patch is older than 3 months, update it to the latest version.

---

Attempt Soft Recovery (Reboots)

Before resetting cluster configurations, attempt to recover quorum using node reboots:

1. Single Node Reboot: Reboot only the node displaying the PAM-UI-1428 error.

2. Simultaneous Reboot: If the single reboot fails, reboot both nodes at the same time. CA PAM uses specific cluster auto-recovery logic during a full cluster outage, which may resolve quorum loss without manual re-clustering.

---

Manual Cluster Reset & Re-configuration

If reboots do not restore cluster quorum, manually isolate and rejoin the nodes:

1. Isolate Nodes:

   • Temporarily remove both Node 1 and Node 2 from the cluster.

2. Reset Configurations:

   • Reset the cluster configuration on both nodes.

   • Log in to each node individually as a super user.

   • Verify full administrative access and confirm that the PAM-UI-1428 error is cleared on both nodes.

3. Re-establish the Cluster:

   • Designate Node 1 as the primary node and generate a new VIP/Cluster Configuration Key.

   • Join Node 2 using the new configuration key and synchronize settings across both nodes.