Performance Management Dashboards with Alarm Views show timeout failure

Products

• CA Infrastructure Management
• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

I'm building dashboards for LAN/WAN/Router/Switch groups, and I've added Alarms as part of the dashboard. we get an error:

"An unexpected error has occurred. Please check the logs for any error messages."

Log message error:

ERROR | Model-Invoker-2          | com.ca.im.portal.plugins.spectrum.models.AlarmViewTableModel     
      | Failed to make request: An exception was thrown while attempting the Web Service call - URL: http://SP:8080/spectrum/restful/alarms
com.ca.im.portal.api.spectrum.rest.SpectrumWebServiceException: An exception was thrown while attempting the Web Service call - URL: http://SP:8080/spectrum/restful/alarms
...
Caused by: java.util.concurrent.TimeoutException

Environment

Release : 3.7

Component : IM Reporting / Admin / Configuration

Cause

Hard coded timeouts on Performance Management are being reached due to slow and inefficient REST calls in Spectrum. Due to the time it takes for the REST calls to return to Performance Management the timeouts strike and the Views fail to render.

Resolution

To fix this now, update Performance Management to r3.7.11 or newer. Starting with the r3.7.11 release the timeout being reached in Performance Management has been both raised by default from 100 to 120 seconds, and also is now configurable. If we still see the issue at 120 seconds it can be raised to avoid failure.

This is the entry from the Fixed Issues list that details the change:

• Symptom: Web service calls to Spectrum for Alarm Console or the subscription alarm service may timeout after 100 seconds.  This usually is due to Spectrum's response time to the request being made.
• Resolution: Updated the timeout to 120 secs, but also made it configurable via the netqosportal.general table or Performance Center admin debug global attributes page.
  • To increase the Alarm Console timeout, run:
    • REPLACE INTO netqosportal.general VALUES('AlarmView_WebService_Timeout', '<timeout in seconds>');
  • To increase the Spectrum alarm status subscription timeout, run:
    • REPLACE INTO netqosportal.general VALUES('SpectrumStatus_WebService_Timeout', '<timeout in seconds>');.
• (3.7.11, DE456697, 20294595

 

Or change the following parameter in http://PC:8181/pc/center/admin/debug/attrs

AlarmView_WebService_Timeout from 120 to 200
SpectrumStatus_WebService_Timeout  from 120 to 200

Additionally with the Spectrum r10.4.2 comes a change to the REST calls available for Performance Management to utilize. They resolve the issue with query performance improvements that avoid the timeouts. A future yet to be determined release of Performance Management will then take advantage of the new Spectrum REST calls to utilize them and further avoid the timeout failures.