Metrics stuck at loading when checking metrics on Aria Operations

Products

• VMware Aria Suite

Issue/Introduction

 

• When checking statistics on Aria Operations, metrics of some objects display "Loading" as shown in the picture below, but eventually showed nothing.
  
  

• When checking statistics on Aria Operations, some timestamps for metrics are several hours later than the check time, and the latest statistical data is missing.
• In /storage/log/vcops/log/analytics-<node id>.log or /storage/log/vcops/log/collector.log, there is "FORWARD_DATA_REGION" error as below:

2024-04-23T07:07:27,414+0000 WARN  [DataForwarder]  com.vmware.vcops.platform.common.DataForwarder.sendData - Failed to send forward data through channel : IRawDataForwarder.NotEnoughBufferSpace: org.apache.geode.cache.CacheWriterException: FORWARD_DATA_REGION region exceed the maximum number (80000) of entries.



Environment

Aria Operations prior to (including) Aria Operations 8.16.2 is affected  

Cause

• Please check the network latency among the cluster to ensure it meets the Aria Operations sizing guide requirements: VMware Aria Operations Sizing Guidelines
• If network latency is fine, this issue can be caused by a large number of communication tasks that overload the analytics, leading to performance and data collection issues. This mechanism has been optimized in Aria Operations 8.17. 

Resolution

This issue has been fixed in Aria Operations 8.17

If the issue occurs with Aria Operations version 8.16.x or earlier, and especially if the log entry 'FORWARD_DATA_REGION region exceeds the maximum number (80000) of entries' is captured in the analytics or collector logs, please upgrade to version 8.17.