The SpectroSERVER may crash when trap forwarding is enabled and the following .vnmrc entry is in use:
This entry disables the rebuilding of the IPAddress cache for SpectroSERVERs that have network connectivity issues.
If the above are true, when there is a disconnect between the Main Location Server and a distributed SpectroSERVER the SS may crash when a trap is received for an IPAddress known by duplicate models. The stack analysis will show the following stack where the actions rem_pim_from_cache and clear_central_cache_contents are seen from the CsAlertFwdMgr function:
#0 0x00007f4a80197ca7 in CsPIMDesc::hash(CsPIMDesc const*) ()
#1 0x00007f4a78d08278 in CsGHashTable::get_node(void const*) const ()
#2 0x00007f4a78d082e1 in CsGHashTable::get(void const*) ()
#3 0x00007f4a7cfaaa76 in CsAlertFwdMgr::rem_pim_from_cache(CsTrapStormDetector*) ()
#4 0x00007f4a7cfab35b in CsAlertFwdMgr::clear_central_cache_contents(CsLscpeHandle const&) () from /opt/SPECTRUM/lib/../SS/libalert.so.1
#5 0x00007f4a7cfab467 in CsAlertFwdMgr::connect_callback(void*, CsError::CsError_e, ConnectionChangeInfo const&) () from /opt/SPECTRUM/lib/../SS/libalert.so.1
#6 0x00007f4a7b7a9b4b in LandscapeConnectionMonitor::doNotificationLoop() const ()
#7 0x00007f4a7fdf9843 in moot_thread_start () from /opt/SPECTRUM/lib/libmoot.so.1
#8 0x0000003e73a438b0 in ?? () from /lib64/libc.so.6
#9 0x0000000000000000 in ?? ()
There was a code flaw in the way the IP address cache was being handled when there were models with the same IP address.
Release: SDBSFO99000-10.2-Spectrum-Device Based Suite-Server FOC
This is tentatively scheduled to be fixed in CA Spectrum release 10.3 due out in late 2017. For Spectrum releases 10.2.0 and 10.2.1 the resolution is to install the following patch on all SpectroSERVERs:
10.2.0 - Spectrum_10.02.00.PTF_10.2.040
10.2.1 - Spectrum_10.02.01.PTF_10.2.112
If you need either patch please open a case withe CA Support online and request the patch.
The release notes show the following:
Symptom: Rare SpectroSERVER crash when Trap Director is enabled and
alert_forwarding_connect_full_flush=false and landscape with duplicate
IP models disconnects and reconnects.
Resolution: Resolved SpectroSERVER crash due to memory corruption.