High Alert Forwarding Queue Latency with Trap Director Enabled

Products

• Network Observability
• Spectrum

Issue/Introduction

High AlertFwd_Queue_Latency occurs during periodic trap bursts or network scans because the default remote forwarding cache timeout triggers frequent distributed searches across all landscapes․​​​​‌​‍‌ High Alert Forwarding Queue latency has been updated․․․․

ERROR MESSAGE: "SSperformance․AlertFwd_Queue_Latency metric indicates periodic high latency"

SYMPTOMS:

• Periodic spikes in Alert Forwarding Queue latency․

• High volume of trap forwarding requests during network scans․

• Remote Forwarding Cache contains numerous entries with model handle 0x0 for local devices․

CONTEXT: Occurs in distributed Spectrum environments using Trap Director (TD) during high-volume trap events or regular network scans․

IMPACT: Increased processing load on SpectroSERVER and potential delays in alert visibility across the landscape․

Environment

DX NetOps Spectrum: Any version

Cause

Regular network scans cause a high volume of traps from devices across the network․ If these devices do not send traps frequently, they age out of the remote cache, forcing Trap Director to perform a distributed search across all landscapes for every incoming trap, creating a processing bottleneck․

Resolution

PREREQUISITES:

• Access to OneClick console with administrative privileges․

• Identification of SpectroSERVERs experiencing queue latency․

STEPS:

1․ EXTEND REMOTE FORWARDING CACHE TIMEOUT: Path: VNM Model > Component Detail > Information > Trap Management

Action: Change the "Remote Forwarding Cache Age Out" value from the default (180 minutes) to 1440 minutes (24 hours)․

EXPECTED: Distributed searches across landscapes are reduced, lowering queue latency․ NOTE: Engineering confirmed no negative impact on CPU, memory, or context switching from this change․

2․ DISABLE UNMANAGED TRAP HANDLING: Action: If trap-based discovery is not required, disable "Unmanaged Trap Handling" in the Trap Management subview․

EXPECTED: Reduced processing load on the VNM model from unknown/unmodeled IP addresses․ NOTE: This does not affect Trap Director functionality but prevents unnecessary event generation for unmodeled devices․

VERIFY SUCCESS:

• Monitor the SSperformance․AlertFwd_Queue_Latency metric over 48 hours․

• Confirm that "bursts" of traps no longer cause significant queue spikes․

• Ensure no negative impact on other SS performance metrics (CPU/Memory)․