Excessive model re-configurations can cause CA Spectrum performance issues
Article ID: 56615
Updated On:
Products
Issue/Introduction
Excessive model re-configurations can cause CA Spectrum performance issues
For most device models, every poll cycle, Spectrum reads the following three attributes:
ifNumber
ifTableLastChange
ifStatusLastChange
If Spectrum detects the value of any one of the above attributes has changed since the last poll and the If_IsAutoCnfgActive attribute 0x11dd4 is enabled, it will automatically reconfigure the model to determine the change and update the model with the change.
This is very expensive in terms of CPU as Spectrum has to read a lot of snmp from the interfaces and ifMIBs.
This can also cause the memory to increase over time.
Spectrum will also log the 0x1001d event (below) which will specify the trigger that caused the model reconfiguration.
Event0001001d:
{d "%w- %d %m-, %Y - %T"} - Device {m} of type {t} has completed an interface reconfiguration. (event [{e}])
Interface Reconfiguration Trigger: {T InterfaceReconfigurationTriggers 1}
Interface Reconfiguration Status: {T CsInterfaceReconfigurationErrors 2}
The following are the possible triggers:
Initial-Reconfiguration - From initial model creation
Forced-Manual-Reconfiguration - Manual reconfiguration
Interface-Stack-Change-Reconfiguration - A change in ifStackLastChange
Interface-Table-Change-Reconfiguration - A change in ifTableLastChange
Interface-Count-Change-Reconfiguration - A change in ifNumber
Auto-Reconfiguration-Activated - When Auto Reconfiguration is enabled after being disabled
SNMP-Contact-To-Device-Re-established - After snmp contact is reestablished after snmp contact was lost
ICMP-SNMP-Contact-To-Device-Re-established - After icmp and snmp contact is reestablished after icmp and snmp contact was lost
Model-State-Changed - When a model state change has been detected
There is an Event Rate rule attached to the 0x1001d event which states if 6 0x1001d events are logged within 31 minutes, Spectrum will log event 0x10050 and assert a minor alarm on the device model.
Event00010050
{d "%w- %d %m-, %Y - %T"} - Device {m} of type {t} has triggered an interface reconfiguration every poll cycle for the past half hour. (event [{e}])
Environment
Release: Any
Component: SPCCSS - SpectroSERVER Performance
Cause
In most cases, the root cause has been devices that dynamically create and remove interfaces that cause updates to the ifNumber, ifTableLastChange or ifStackLastChange attributes.
Resolution
Interface Reconfiguration can be handled Automatically within Spectrum Self-Health submenu.
- Excessive interface re-configurations handling:
Previously SS raised ‘Excessive Interface Reconfigure' alarm on a device if six interface re-configurations occur within 31 minutes. This frequent interface re-configurations adversely affect the SpectroSERVER's performance. To address this issue, from 10.3.2 onwards, if this option is enabled, SS disables auto-reconfiguration on such devices and waits for an age-out time (configurable to 1 hour by default). After the age-out time expires, SS enables the reconfiguration.
The interface reconfiguration functionality can be manually disabled by doing the following:
1. Setting If_IsAutoCnfgActive attribute 0x11dd4 to No. This will disable all interface reconfigurations for any reason. This can be set in any of the following ways:
a. In the Reconfiguration subview in the Information tab of the model by setting "Automatically Reconfigure Interfaces" to "No".
b. In the Attributes tab of the model by filtering for and selecting the If_IsAutoCnfgActive attribute 0x11dd4 to "No".
c. In the Attribute Editor by selecting "Automatically Reconfigure Interfaces" in the "Change Management" folder
d. Using the Command Line Interface (CLI) update command. Reference the "Command Line Interface User Guide" for information on using CLI.
2. If the Interface Reconfiguration Trigger is "Interface-Stack-Change-Reconfiguration", set the Use_If_Stack_Last_Change attribute id 0x130bc to "False" on the model using the Attributes tab, Attribute Editor or CLI.
3. If the Interface Reconfiguration Trigger is "Interface-Table-Change-Reconfiguration", set the Use_If_Table_Last_Change attribute id 0x11f7f to "False" on the model using the Attributes tab, Attribute Editor or CLI.
Feedback
