Prediction Engine queue grows and fails to drain after CU6 upgrade - DX UIM

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

Introduction

After upgrading to DX UIM 23.4 CU6, the prediction_engine.QOS_MESSAGE queue grows rapidly and does not drain. This typically occurs in environments with high QoS volume or where Time-To-Threshold (TTT) is configured.

Symptoms

• The prediction_engine.QOS_MESSAGE queue turns yellow in the Hub GUI.
• prediction_engine.log repeatedly shows: INFO calculator.DataGathererImpl - TTT definition not found in memory, attempting to reload from file for metric: ####
• hub.log contains errors failing to open .sds files for the prediction engine queue.
• High CPU or memory usage by the prediction_engine probe.

Environment

• DX Unified Infrastructure Management 23.4 CU6
• prediction_engine probe 23.4.6

Cause

A defect in the prediction_engine causes repeated attempts to reload TTT definitions, overwhelming the queue processing. Additionally, default Java memory settings for the probe may be insufficient for high-volume environments.

Resolution

Step 1: Increase Probe Memory

Increase the memory allocation for both baseline_engine and prediction_engine to prevent performance degradation.

1. Open Raw Configure for the probe.
2. Navigate to the setup section.
3. Update the following keys:
   • java_mem_init: 2048m
   • java_mem_max: 4096m
4. Deactivate the probe, wait for it to stop, then activate it.

Step 2: Apply Patch or Upgrade

• Workaround (CU6): Apply the prediction_engine-23.4.6-defect-DE661563_test_fix patch. (attached to this KB)
• Permanent Fix: Fixed in release CU7 and higher. See Download Broadcom products, patches and software for steps to download the latest release.

 

Additional Information

For additional information  Contact Support - Support Portal - Broadcom support portal

For defects and enhancements, subscribe to this article to be updated on fix status. Subscribe to a Broadcom knowledge article by article or product