Getting error "The queue 'data_engine' is 900 MB. Check the hub configuration."

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

We are getting the error "The queue 'data_engine' is 900 MB. Check the hub configuration." frequently now a days but we could see the queue size is below 100MB. 

Environment

Release: DX UIM 20.4*/23.4

Component: Data Engine

Resolution

The growing queue would be an indication that you are collecting QOS data faster than you can write it out to Oracle.

 

We use SQL Server but for reference, here are the values that we use that might impact your processing speed:

<setup>
    data_management_active = yes
    data_management_time_spec = RRULE:FREQ=DAILY;INTERVAL=1;BYHOUR=20;BYMINUTE=00
    data_management_timeout = 172800
    auto_reindex = no
    hub_bulk_size = 2000
    data_management_compress = yes
    delete_raw_samples = 60
    delete_history_samples = 120
    raw_data_extra = 15
    number_of_subpartitions = 5
    data_management_partition = yes
    min_free_space = 10
    monitor_interval = 300
    idle_timeout = 900
    log_bulk_stats = 0
    log_inserted_rows = 1
    log_lsv_rows = 0
    table_maintenance_mode = 0
    table_maintenance_online_mode = 0
    table_maintenance_loglevel = 6
    statistics_age = 24
    threads = 0
    thread_count_insert = 32
    update_metric_id = yes
</setup>

 

I'd suggest working with thread_count_insert as a starting point. From experience this value should be larger than the number of RN tables that are actively getting written to frequently. It will use a login per thread so you might need to increase that on the database engine but you can get significant improvement in speed making this value larger.