hub prediction_engine or baseline_engine QOS_MESSAGE queue turned yellow and is backing up
search cancel

hub prediction_engine or baseline_engine QOS_MESSAGE queue turned yellow and is backing up

book

Article ID: 73563

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM) Unified Infrastructure Management for Mainframe CA Unified Infrastructure Management On-Premise (Nimsoft / UIM) CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

A hub's baseline_engine or prediction_engine QOS MESSAGE queue appears yellow and is backing up.

One of our hubs has a queue that is backing up and not draining. As per the hub Status Tab window, its baseline_engine queue baseline_engine.QOS_MESSAGE turns yellow and doesn't send any messages.   (Can also impact prediction_engine)

  • emptying queue makes no difference
  • restarting probe does not help
  • redeploying probe does not help
  • recheck of queue does nothing
  • baseline_engine probe security is set properly (admin *)

We see messages like this in the baseline_engine log, but we do not see a corresponding message for baseline_engine.QOS_MESSAGE:

[ Thread-5] INFO queuemessengerimpl.QueueReaderImpl - Queue baseline_engine.BASELINE_CONFIG is active

Environment

  • DX UIM - all versions
  • baseline_engine probe deployed and actively collecting baseline data, or
  • prediction_engine probe with Time-To-Threshold alarms enabled

Cause

  • Unknown

Resolution

  1. Deactivate the baseline_engine (or prediction_engine) probe on the remote hub

  2. Delete both baseline_engine queues (baseline_engine.QOS_MESSAGE and baseline_engine.QOS_MESSAGE) via the hub GUI Queues Tab window and click Apply  (Or, delete the prediction_engine.QOS_MESSAGE queue).

  3. Activate baseline_engine (Note that the baseline_engine.BASELINE_CONFIG is created automatically and does not need to be created/recreated manually). Check to be sure its created via the hub Status Tab window.

  4. Choose any robot under the remote hub with the issue

  5. Login to Admin Console, and go to the robot you chose in 4.

  6. Select the Probes tab, and configure any of them. For instance, cdm probe or processes probe, or any other you prefer. For instance, I will select cdm probe for this example.

  7. In the probe configuration, select a monitor, and on right panel check the "Compute Baseline" check box. Then click the Save button.

 

 

7. After a few minutes you should see the baseline_engine.QOS_MESSAGE queue show up in the hub Queues and Status Tab windows. The queues should be green and successfully processing messages.

Note: for Prediction Engine, instead of choosing "Compute Baseline", locate an existing Time-To-Threshold alarm, uncheck the box and check it again, then click OK to save.

 


Note that the 'bulk_size' for both queues should show their bulk_size as 1000 (this is their default out of the box bulk_size setting).

 

Powered by Wolken Software