probe_discovery queue stuck does not drain and queues up

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The probe_discovery queue is stuck and continues to grow, reaching over 500,000 queued messages while processing very few. 

Environment

• DX UIM 23.4CU8 and Earlier

Cause

High frequency of device attribute changes (multiple MAC/IP addresses) triggering excessive re-correlation events and a product defect in how discovery_server handles these recreation events can cause these issues in Large Scale Environments. 

Resolution

Deploy Discovery Server Patch

Install the discovery_server_23.4.3.4.zip patch which contains code enhancements to handle re-correlation events more efficiently. This fix ensures that incoming devices with high attribute volatility do not impact overall probe performance.

 

NOTE this patch is valid for DX UIM 23.4CU4. 

• If delays persist, adjust the connection pool settings in the discovery_server raw configuration:
  • setup > db_connection_pool_max_idle = 50
  • setup > db_connection_pool_size = 50

Additional Information

Related KB: 

• Discovery server probeDiscovery queue is backing up displays as yellow and not working as expected

NOTE: 

The fix is NOT present in DX UIM 23.4.CU5/CU6/CU7/CU8 GA. The will be included in CU9. If a fix is needed for pre CU9, Contact Broadcom support