"TSDB Server failed to flush data to HBase" issue observed in VCF Operations for Networks

Products

• VCF Operations for Networks

Issue/Introduction

Observe infrastructure health alerts in the VCF Operations for Networks user interface indicating that the TSDB Server is unable to process data. This issue typically occurs when the Time Series Database (TSDB) encounters a hang during the flush process to the HBase layer, even if individual services report as healthy.

• The following error is observed on UI:
  
• In the UI under Settings > Infrastructure and Support, the following message appears: TSDB Server failed to flush data to HBase.
• System health checks indicate services are Running, but data is not being updated.
• Infrastructure health status is reported as Poor.
• Running the following command shows that all services are healthy or not:
  

  ./run_all.sh sudo /home/ubuntu/check-service-health.sh -p -d

Environment

VCF Operations for Networks 6.x

Cause

The TSDB service may fail to flush data to HBase due to service inconsistencies following a restart or fresh deployment.

Resolution

 TSDB-server service needs to be restarted using the command below on Platform 1 after switching from the support user to the ubuntu user:

ub
/home/ubuntu/run_all.sh "sudo systemctl stop tsdb-server.service"
/home/ubuntu/run_all.sh "sudo systemctl start tsdb-server.service"

Additional Information

• For instructions on downloading updates, see Download Broadcom Products and Software.
• To speak with a customer representative or a Support Engineer, see Contact Support.
• Subscribe to this article to receive updates on fix status: How to Subscribe to a Knowledge Document.