data_engine queue not emptying and seems slow to drain

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)
• CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

Yesterday we could perform recommended upgrade on open case 32917086 (Upgrade all hubs to hub_9.33_HF1.zip and Upgrade all hub robots to robot_update_9.33_HF6.zip).

Everything is working fine but this morning checked that data engine queue have lot of data queued vs sent data.

 

Refreshing some times later, seems like working properly:

Is this a normal behavior? Queue takes some time to stabilize?

Environment

Release : 20.3

Component : UIM - DATA_ENGINE

Cause

- data_engine configuration

Resolution

data_engine configuration updated with best practice settings.

Settings changed:

1. hub_bulk_size should be no higher than 1750
2. thread_count insert from 0 to 24 (despite actual # of virtual processors)
3. loglevel set to 5 and logsize from 100 to 50000 for troubleshooting, then post-troubleshooting set loglevel to 0
4. data_management_timeout from 7200 to 65536
5. indexing should remain disabled and partitioning should be enabled
6. manual partitioning of the largest tables must be completed first before enabling partitioning option in the data_engine