probe_discovery queue stuck does not drain and queues up

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The probe_discovery queue is stuck and continues to grow, reaching over 500,000 queued messages while processing very few. 

Environment

DX UIM 23.4CU7 and Earlier

Cause

High frequency of device attribute changes (multiple MAC/IP addresses) triggering excessive re-correlation events and a product defect in how discovery_server handles these recreation events can cause these issues in Large Scale Environments. 

Resolution

Deploy Discovery Server Patch

Install the discovery_server_23.4.3.4.zip patch which contains code enhancements to handle re-correlation events more efficiently. This fix ensures that incoming devices with high attribute volatility do not impact overall probe performance.

 

NOTE this patch is valid for DX UIM 23.4CU4. 

• If delays persist, adjust the connection pool settings in the discovery_server raw configuration:
  • setup > db_connection_pool_max_idle = 50
  • setup > db_connection_pool_size = 50

Additional Information

Related KB: 

• Discovery server probeDiscovery queue is backing up displays as yellow and not working as expected

NOTE: 

The fix is NOT present in DX UIM 23.4.CU5/CU6/CU7/CU8 GA. The will be included in CU9. If a fix is needed for pre CU9, Contact Broadcom support