qos_processor not updating origins - error - Failed to enqueue pending qos object, queue capacity exceeded.

Products

• DX Infrastructure Management
• NIMSOFT PROBES

Issue/Introduction

Origin information in S_QOS_DATA is not being updated or enriched by the qos_processor probe.

The log file contains messages similar to the following:

Aug 11 16:14:20:519 [Qos Monitor Enricher #5, qos_processor] Failed to enqueue pending qos object, queue capacity exceeded. The update will be made when memory is released. No data will be lost. S_QOS_DATA { qos_name: QOS_PROCESS_CPU, source: xxxxxxx, target: yyyy, nim_origin: YOUR_ORIGIN, origin: YOUR_ORIGIN, modifier: nimsoft, host: 1.2.3.4, robot: uim_robot_name, probe: processes, table_id: null, qos_def_id: null } 

 

 

Cause

This can occur in an environment where a large amount of QoS data is coming in, and the probe is unable to keep up with the incoming volume due to a memory constraint.

Environment

Release: CNMSPP99000-8.47-Unified Infrastructure Mgmt-Server Pack-- On Prem
Component:

Resolution

1. Open "Raw Configure" on the qos_processor probe.

2. Navigate into the "setup" folder/section.

3. Click "New Key..."

4. Add a new key called: database-update-queue-capacity

5. Set a value for this key of 20000  (the default value is 10000, so doubling it is a good place to start.)

 

If the error message persists after this, you may need to set the value even higher, such as 50000.  The qos_processor may consume more memory with this value increased, so it is a good idea to increase the allocated memory for the probe as well.  To do so:

1. Use "Raw Configure" on the qos_processor probe

2. Navigate to the "startup" folder, then "opt" underneath that.

3. Increase the value set for "java_mem_max" here.  Default is 2048m (2048 megabytes.)   If you double the above value (e.g. 20000) it would be a good idea to double the memory allocation here as well, although it may be advisable to monitor the probe's memory usage and set the value according to your own observations.