data_engine best practices - settings, performance optimization, and discussion of Database Server Best Practices

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)
• CA Unified Infrastructure Management On-Premise (Nimsoft / UIM)
• CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

Potential Benefits to be derived

- Enhance and/or Sustain 24x7 QOS metric data collection!
- Ensure the health of the data_engine and data processing
- Application of Database Server Best Practices
- Optimize QOS data processing performance
- Achieve the Goal of Lights-out data management that lasts more than a few years
- Keep Data Administration and Maintenance to a bare minimum
- data_engine knowledge sharing
- Provide feedback to Development for any new issues or improvements

Environment

• Release: 20.4
• Component: UIM - DATA_ENGINE

Cause

• Proactive engagement

Resolution

Summary

Fine-tuning of data_engine, discussion of DB Best Practices, improved data_engine throughput, performance optimization, and more.

Changed and adjusted hub postroute_reply_timeout from default of 180 to 300.
This value is also in seconds, and determines how long the hub will wait for a reply from any queue/subscriber after sending messages. It controls how long the hub waits (in seconds) for a reply from the remote hub after sending a bulk of messages on a queue before deciding that it didn't go through and then resends the bulk.

Checked and improved data_engine throughput

Based on current settings...

Changed hub_bulk_size in the data_engine from the current setting of 1000 to 1750.
Waited up to 20 minutes...
After repeated inspection during the Webex session, we saw that avg. throughput was increased by +20,000 messages from avg of approx. 41k, up to 65k-ish.

 

Checked partitioning

- data retention is being honored.

 

Large Tables

Checked large tables and the largest was 323M. There were approx. 50 tables over 10M.

 

S_QOS_DATA (a.k.a. the QOS Object 'Index')

5M rows. There are no known issues related to this size but duly noted and at some point, the customer may want to get rid of any orphaned data. We have a process to do that but it's currently not necessary. In some customer environments, >1M rows may slow down queries related to Operator Console, Reports, and Dashboards.

Overall health check for data_engine and DATA management/administration completed.

========================================================================