Forcing the Replica node to take over the Primary role in a VMware Aria Operations (formerly vRealize Operations)cluster
search cancel

Forcing the Replica node to take over the Primary role in a VMware Aria Operations (formerly vRealize Operations)cluster

book

Article ID: 315910

calendar_today

Updated On:

Products

VCF Operations/Automation (formerly VMware Aria Suite) VMware vSphere ESXi

Issue/Introduction

Following a cluster reboot in VMware Aria Operations (formerly vRealize Operations), the cluster nodes experience an unexpected role reversal:
  • The original Primary node is acting as a Replica.
  • The original Replica node is acting as the Primary/PostgreSQL master.
  • Executing the following command within a root SSH session on the current replica node does not return any rows
    su - postgres -c "/opt/vmware/vpostgres/current/bin/psql -d vcopsdb -p 5433 -x -c 'SELECT * FROM pg_stat_replication;'"
  • Executing the following command within a root SSH session on the current replica node shows streaming status
    su - postgres -c "/opt/vmware/vpostgres/current/bin/psql -d vcopsdb -p 5433 -x -c 'SELECT * FROM pg_stat_wal_receiver;'"

This article provides steps to force the Replica node to take over the Primary role in a VMware Aria Operations (formerly vRealize Operations) Operations cluster.

Environment

VMware Aria Operations 8.x

Cause

 In a High Availability (HA) cluster, if the Primary node becomes unavailable (e.g., during a power cycle or reboot), the Replica node is designed to take over the PostgreSQL master role automatically to ensure the cluster remains operational. This is expected behavior.

Resolution

  1. Take snapshots of the nodes (Primary/Primary Replica/Data) as per KB Snapshot Creation in VMware Aria Operations

  2. Log in to the Admin UI on the Primary node using the URL: https://IP_or_FQDN/admin
    Note: Replace IP_or_FQDN with the IP address of FQDN of the Primary node.

  3. After the snapshot process is done bring the cluster back Online.

  4. Select the node currently listed as Primary (this should be the node you wish to demote to Replica) and click Take Node Offline.

  5. Provide a reason and click OK.

  6. Wait approximately 15 minutes. The HA subsystem will promote the remaining online node (your original Primary) to the Primary role.

  7. Once the UI reflects the role change, select the offline node and click Bring Node Online.

  8. The node will rejoin the cluster and assume the Replica role.

  9. Verify the status of both nodes shows Healthy.