Spectrum is generating 0x10d00 and 0x10d0c events at the same time for Cisco devices every hour

Products

• Network Observability
• Spectrum

Issue/Introduction

We noticed that the 600 bigger Cisco SNMPv3 devices are not reachable via SNMP at the same time every hour, but also reachable again in the same second. The point of the hour differs per device. Here is an example where it always at minute 4. For other devices the minute is different but the same every hour.

This causes a huge amount of events, because each port and application also produces many events. We have about 10,000 events in 5 minutes.

Environment

Spectrum 24.3.6

Cause

There is an incorrect configuration in the device's VRF configuration.

The devices are throwing VRF authorization error when we tried with 'spectrum' context name by using sapwalk2 tool (no Spectrum application involved)

And the other context name 'spectrum-IPv6' didn't exist on the device.

Resolution

The device administrator changed the configuration:

snmp-server group <groupname> v3 auth context spectrum

snmp-server group <groupname> v3 auth context spectrum-IPv6

snmp-server context spectrum

snmp-server context spectrum-IPv6 

After that, the events disappeared.

Additional Information

The cContextMappingVrfName (0x213f5a) attribute ID has two Instances:

Instance ID: 8.115.112.101.99.116.114.117.109   (convert ASCII to Text) ==> spectrum

Instance ID: 13.115.112.101.99.116.114.117.109.45.73.80.118.54   (convert ASCII to Text) ==> spectrum-IPv6

 

The device is not responding to OID: 1.3.6.1.2.1.4.22.1.2 (ipNetToMediaPhysAddress) for spectrum context. And the spectrum-IPv6 context is unknown: 

sapwalk2.exe -i <IP address> -v v3 -s 1.3.6.1.2.1.4.22.1.2 -u <user> -l AnP -xt SHA -xa ******** -xn spectrum

#Agent is broken. Received error[(16)Authorization Error] other than NO_SUCH_NAME. Making sure no more oids left. Trying 1.3.6.1.2.1.4.22.2

 

sapwalk2.exe -i <IP address> -v v3 -s 1.3.6.1.2.1.4.20.1.1 -u <user> -l AnP -xt SHA -xa ******** -xn spectrum

#Agent is broken. Received error[(16)Authorization Error] other than NO_SUCH_NAME. Making sure no more oids left. Trying 1.3.6.1.2.1.4.20.2

 

Sapwalk2 also fails to retrieve data from the following OIDs:

• 1.3.6.1.2.1.4.20.1.1 (ipAdEntAddr reference info)
• 1.3.6.1.2.1.4.20.1.2 (ipAdEntIfIndex reference info)

sapwalk2.exe -i <IP address> -v v3 -s 1.3.6.1.2.1.4.20.1.2 -u <user> -l AnP -xt SHA -xa ******** -xn spectrum

#Agent is broken. Received error[(16)Authorization Error] other than NO_SUCH_NAME. Making sure no more oids left. Trying 1.3.6.1.2.1.4.20.2

 

sapwalk2.exe -i <IP address> -v v3 -s 1.3.6.1.2.1.4.22.1.2 -u <user> -l AnP -xt SHA -xa ******** -xn spectrum-IPv6

Error: Received unexpected Report PDU[1.3.6.1.6.3.12.1.5.0(snmpUnknownContexts.0)]. Walk stopped.