Metrics stuck at loading when checking metrics on Aria Operations

Products

• VMware Aria Suite

Issue/Introduction

• When checking statistics on Aria Operations, metrics of some objects display "Loading" as shown in the picture below, but eventually showed nothing.
  
  
• When checking statistics on Aria Operations, some timestamps for metrics are several hours later than the check time, and the latest statistical data is missing.
• In /storage/log/vcops/log/analytics-<node id>.log or /storage/log/vcops/log/collector.log, there is "FORWARD_DATA_REGION" error as below:
2024-04-23T07:07:27,414+0000 WARN  [DataForwarder]  com.vmware.vcops.platform.common.DataForwarder.sendData - Failed to send forward data through channel : IRawDataForwarder.NotEnoughBufferSpace: org.apache.geode.cache.CacheWriterException: FORWARD_DATA_REGION region exceed the maximum number (80000) of entries.

Environment

Aria Operations prior to (including) Aria Operations 8.16.2 is affected.

Cause

• Please check the network latency among the cluster to ensure it meets the Aria Operations sizing guide requirements: VMware Aria Operations Sizing Guidelines
• If network latency is fine, this issue can be caused by a large number of communication tasks that overload the analytics, leading to performance and data collection issues. This mechanism has been optimized in Aria Operations 8.17.x.

Resolution

• This issue has been fixed in Aria Operations 8.17.x.
• If the issue occurs with Aria Operations version 8.16.x or earlier, and especially if the log entry 'FORWARD_DATA_REGION region exceeds the maximum number (80000) of entries' is captured in the analytics or collector logs, please upgrade to version 8.17.x.