Events and Alarms missing from Spectrum Report Manager reports

Products

• CA Spectrum
• DX NetOps

Issue/Introduction

Event sync between Archive Manager & SRM very slow with CORBA "DOES_NOT_EXIST" exception found in Spectrum Tomcat log

After running repmgrInitalizeLandscape.bat script to Initialize the SRM database, SRM may be polling the landscapes for Events, however the polling is taking a very long time. Checking the Last Event Time on OneClick Report Manager configuration web page for "Spectrum STATUS" shows this time increase at a very slow rate.

The Spectrum Status page for SRM shows landscapes that are not up to date for Event sync and polling.

These exceptions are seen in the $SPECROOT/tomcat/logs/catalina.out log file for impacted landscapes. Each impacted landscape would be referenced where <landscapeName> is seen below, with the error appearing at least once for each impacted landscape.

Sep 26, 2017 01:26:18.361 (SRM/LandscapeManager/LandscapeThread_0) (SRM_Events) - (ERROR) - Expection occurs while retrieving events for <landscapeName>. retrying in cycle Caused by: exception com.aprisma.spectrum.core.idl.CsCException.CsCSpectrumException { 
com.aprisma.spectrum.core.idl.CsCError.CsCError_e error=DOES_NOT_EXIST,
java.lang.String messageFormatID="CsCEventDomain.getNextEventListByAttrFilter.requestID.DOES_NOT_EXIST", 
com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue[] messageValueList={

union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { java.lang.String textString="CsEventDSrvc.cc" },
union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { java.lang.String textString="%I% %G% %U%" },
union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { int intValue=958 },
union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { java.lang.String textString="<landscapeName>" },
union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { int intValue=0 },
union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { java.lang.String textString="<landscapeName>/CsCEventDomain" },
union com.aprisma.spectrum.core.idl.CsCAttribute.CsCValue { int intValue=100061 }} 
} 
at com.aprisma.spectrum.core.idl.CsCException.CsCSpectrumExceptionHelper.read(CsCSpectrumExceptionHelper.java:40) 
at com.aprisma.spectrum.core.idl._CsCEventDomainStub.getNextEventListByAttrFilter(_CsCEventDomainStub.java:1457)
at com.aprisma.spectrum.app.repmgr.dc.event.EventDataModel.getEvents(Unknown Source) 
at com.aprisma.spectrum.app.repmgr.dc.event.EventPoller.doHandleEvents(Unknown Source)
at com.aprisma.spectrum.app.repmgr.dc.event.EventPoller.handleEvents(Unknown Source)
at com.aprisma.spectrum.app.repmgr.dc.event.EventPoller.initEventHandling(Unknown Source)
at com.aprisma.spectrum.app.repmgr.dc.landscape.LandscapeManager.setupEventPoller(Unknown Source)
at com.aprisma.spectrum.app.repmgr.dc.landscape.LandscapeManager.access$100(Unknown Source)
at com.aprisma.spectrum.app.repmgr.dc.landscape.LandscapeManager$LandscapeThread.run(Unknown Source)

Environment

All supported DX NetOps Spectrum releases

Resolution

To resolve this take the following steps.

1. On OneClick server edit the $SPECROOT/tomcat/webapps/spectrum/WEB-INF/web.xml file.
2. Find the following lines within the file: 
   
   <context-param> 
   <param-name>com.aprisma.spectrum.app.repmgr.dc.event.ForceEventPolling</param-name>
   <param-value>false</param-value>
        <description>
              Enabling Event polling forcefully after CORBA DOES_NOT_EXIST exception
        </description>
   </context-param>
   
   
3. Change false (default) to true as seen here:
   
   <param-value>true</param-value>
   
4. Save the file changes.
5. Restart the Spectrum Tomcat service.
   

Once the changes are completed monitor the $SPECROOT/tomcat/log/catalina.out. Do those same errors continue?

If the errors do not continue to appear, confirm Events begin returning to SRM where they were previously missing.

Additional Information

Event Storage Best Practices