Alert: "vRealize Operations Cluster processes may not have enough memory"

Products

• VCF Operations/Automation (formerly VMware Aria Suite)
• VCF Operations

Issue/Introduction

• Aria/VCF Operations UI becomes very slow and the node starts consuming a lot of CPU.
• Cluster is CA enabled (Continuous Availability)
• This may self resolve after some time, or may require that the cluster is brought down and back up to remediate.
• During this time, the metric 'Garbage collection GC workload (%)' will spike to 100%, and the metric 'Elastic Heap Memory Remaining (%)' drops to 5% for the Aria//VCF Operations master nodes object.
• Heapdump files may be generated by these events on the Aria/VCF Operations primary node.
• After some time Aria/VCF Operation node starts consuming too much CPU and leaves the cluster inoperative.
• The Garbage Collection | GC Workload (%) shows high on affected node(s).

Environment

• Aria Operations 8.18.x
• VCF Operations 9.0.x

Cause

This is a known issue with Gemfire service in Aria Operations 8.18.x and VCF Operations 9.0.x

This issue have predominantly been reported in CA clusters where the fault domains are in geographically different locations.

Gemfire is extremely sensitive to high latency, which may result in this issue being observed.

Resolution

Fixed in VCF Operations 9.1 and higher. See Download Broadcom products and software to download this release.

Additional Information

Upgrade sequence to 9.1