While Running a Large Discovery MAL Alarms Generated on a Large Number of Devices

Products

• Network Observability

Issue/Introduction

Using the discovery configuration to run a large discovery againt existing devices to refresh them it was noticed that a group of devices started to 
    generate Management Agent Lost alarms (MAL). Manual polling of these devices then failed until the SNMPv3 cache was cleared on the SpectroSERVER

Cause

The dicovery configuration contained a large number of SNMP v3 profiles to use during the discovery. It was found that the v3 cache was being updated with
     an entry for each v3 profile tried for the device being discovered. This in turn caused polling from the SS to fail.

When the snmpv3 cache was dumped to the VNM.OUT it was noted that there were now multiple profiles associated for each profile being discovered rather than
     just the one that was used for polling. Clearing the v3 cache (CLI Action code, Reset SNMP v3 Authentication (right click menu) or SS restart) then allowed the
     SpectroSERVER to regain snmp communication.


example seen in the cache dump for one device

----------------------------------------------------------------------------- -------------------SNMP V3 REMOTE PROFILE DATA ------------------------------ ----------------------------------------------------------------------------- Remote User Profile for IP : xxx.xxx.xxx.xxx  ----> username : SnmpTestUser   ----> dh_v3_profile: 0   ----> Engine ID: aa:bb:cc:dd:ee:ff:gg:hh:ii  ----> Engine BOOT: 2664394   ----> Engine TIME: 773   ----> Engine TIME Last updated at: 1761757491  Remote User Profile for IP : xxx.xxx.xxx.xxx  ----> username : v3UserName   ----> dh_v3_profile: 0   ----> Engine ID: aa:bb:cc:dd:ee:ff:gg:hh:ii  ----> Engine BOOT: 2664394   ----> Engine TIME: 773   ----> Engine TIME Last updated at: 1761757491  Remote User Profile for IP : xxx.xxx.xxx.xxx  ----> username : v3UserName   ----> dh_v3_profile: 0   ----> Engine ID: aa:bb:cc:dd:ee:ff:gg:hh:ii  ----> Engine BOOT: 2664394   ----> Engine TIME: 773   ----> Engine TIME Last updated at: 1761757491  Remote User Profile for IP : xxx.xxx.xxx.xxx  ----> username : v3UserName   ----> dh_v3_profile: 0   ----> Engine ID: aa:bb:cc:dd:ee:ff:gg:hh:ii  ----> Engine BOOT: 2664394   ----> Engine TIME: 773   ----> Engine TIME Last updated at: 1761757491  Remote User Profile for IP : xxx.xxx.xxx.xxx  ----> username : v3UserName   ----> dh_v3_profile: 0

Resolution

Modify the SpectroSERVER .vnmrc file and add the configuration entries blow if they do not already exist. 


$SPECROOT/SS/.vnmrc
delete_invalid_empty_profiles=true
delete_unknown_report_type_profiles=true
delete_invalid_profiles=true


The settings will take effect on a SpectroSERVER restart (which also clears and rebuilds the v3 cache)

Additional Information

Dump the SNMPv3 cache to the VNM.OUT log (via CLI)

cd $SPECROOT/vnmsh/
./connect
./update action=0x10331 mh=<landscape handle ex. 0x1000000>


Clear the entire SNMPv3 cache (via CLI)

cd $SPECROOT/vnmsh/
./connect
./update action=0x10333 mh=<landscape handle ex. 0x1000000>