No metric data is displayed for flows filter by 1 day last week in Aria Operations for Networks GUI

Products

• VCF Operations for Networks

Issue/Introduction

Error on the GUI the Aria Operations for Networks: " TSDB server failed to flush data to HBASE." 

or

A search of flows for a particular vm filtered by "last week", graph displays continuous data every day but when same search is filtered for a custom 24 hour period for a day in that week more than 1 day ago (i.e. not yesterday) then no data is displayed.

flows where destination vm = <vm name> order by session desc

Refer to screenshots as below:

Flows filtered by "Last week" displays continuous data on the graphs as per screenshot below:

Flows filtered by custom 24 hour time period earlier than yesterday displays no data on the graphs as per screenshot below:

 

Environment

Aria Operations for Networks 6.12
Aria Operations for Networks 6.12.1
Aria Operations for Networks 6.13
Aria Operations for Networks 6.14

Cause

• In restapilayer log on Platform1, the following message is seen:
  
  

  2025-02-21T20:51:44.774Z INFO vnera.metricseval.MetricsHelper recursive-batch-operation-exec-9 isDataAvailableInTsdb:1417 Oldest data in tsdb at 1734062400000

  
  
  1734062400000 = Dec 13 2024 04:00:00 (should not be older than last 24 hours).
  
  This points to the fact that the tsdb-server is not cleaning the old data properly.
  
  
• tsMapGC in tsdb-server is not running on Platforms 15, 10, 8 and 6. These platforms have been in the server initializing state for a long time.
  
  Oldest tsdb-server logs:

  2025-02-10T20:37:26.542Z WARN tsdb.impl.TsdbServerImpl tsdb-server-exec-42 getChanges:806 rejecting request as server is initializing.

  
  
  
• This is happening due to the following issue: the number of actions is not decreasing.
  
  

  2025-02-10T20:37:28.847Z INFO hbase.client.AsyncRequestFutureImpl put-3-queue-worker waitUntilDone:1175 _4, waiting for 5339  actions to finish on table: metrics.1800

  
  
  

Resolution

tsdb-server needs to be restarted on the all platform nodes to resolve the issue.

1. SSH into Platform node 1 using the support user.
2. Run the following commands on Platform node 1:

 

ub
/home/ubuntu/run_all.sh "sudo systemctl stop tsdb-server.service"
/home/ubuntu/run_all.sh "sudo systemctl start tsdb-server.service"