SpectroSERVER shutting down soon after startup trying to allocate XXX bytes of memory
The $SPECROOT/SS?VNM.OUT is showing the following error messages:
Jul 18 13:16:17 ERROR TRACE at VNM.cc(797): S:/CA/SPECTRUM/SS/SpectroSERVER.exe is out of memory while allocating 11 bytes. Scheduling shutdown.
Jul 18 13:16:20 ERROR TRACE at VNM.cc(841): C:/SPECTRUM/SS/SpectroSERVER.exe is out of memory while allocating 12 bytes. Continuing shutdown.
CA memory allocation error. Aborting.
In this instance, the cause of this excessive memory consumption in a very short time is due to an excessive number of traps that are being sent to the SpectroSERVER. Spectrum does have trap storm alarming and suppression built in, but Spectrum, as designed, must first read the traps, which uses memory, in order to know that it is part of a trap storm and not a valid trap. This causes the memory exhaustion and will eventually use up all of the free memory and Spectrum will shutdown as a result.
Any Spectrum version
You will need to locate the devices) that are sending an excessive number of traps and resolve the problem that causes them to send the traps to Spectrum.
If it is not known or it is not possible to resolve this problem one option is to temporarily disable all trap processing on the SpectroSERVER.
You can do this by editing the .vnmrc file ($SPECROOT/SS) and changing the option:
This will allow the SpectroSERVER to start up as it is not processing traps.
Alternatively you can also change the SNMP Trap Port number in the .vnmrc file
This will also stop Spectrum from receiving traps on the default port of 162.
Once the devices have been identified that are sending excessive traps, you can stop the SpectroSERVER and change back these settings to the original values of empty (default).
It is recommended to use Wireshark (Windows) or tcpdump (Linix) on the SpectroSERVER system to examine the trap ports and locate where the traps are coming from.