Many a times we compare the monitoring data on cluster members in an incorrect way, leading to a wrong conclusion on the replication status between the servers.
This article describes the correct way to monitor data on cluster members using Cluster Replication Status window.
Before monitoring the replication data for cluster members you want to compare, you must log into Symantec Encryption Management Server web interface in a separate Web browser window for each cluster member.
For example, if you have two cluster members A and B, you want to check the replication rates of two cluster members, you must log into two instances of Symantec Encryption Management Server.
[A] Compare the replicated data from cluster member A to cluster member B.
In the picture below we see that server A data is in Sync with server B.
[B] Compare the replicated data from cluster member B to cluster member A.
In the picture below, we see that server B data is not in sync with server A.
On server B clustering, server B's data have higher change ids than in server A clustering page, which indicates that there is some data on server B which is not replicated on server A.
Additionally, there are some read-only commands that can be used to check replication on the backend via SSH:
To verify connectivity issues within cluster members:
To verify the incoming and outgoing connections of a node:
pgprepctl debug list
Used to monitor the queues and make sure replication is working:
Review the highwatermarks from each member which will indicate whether the cluster is up or not. When the hightwatermarks are around the same values then the cluster health is good.
At times it is normal to see the highwatermarks differences when the cluster is under load during peak periods but, will converge once the traffic is low.