Alarm sync between CAPM and Spectrum stops when primary spectro server goes down

book

Article ID: 131516

calendar_today

Updated On:

Products

CA Performance Management - Usage and Administration

Issue/Introduction

When a primary SpectroServer is stopped and the secondary takes over, there are no more alarms from CA PM to spectrum. Even when the primary SpectroServer is back up and running, there are still no alarms with the following errors appearing repeatedly in the tomcat log:

Jan 16, 2019 15:10:48.120 (http-nio-80-exec-5) (NetqosSyncLog) - (ERROR) - Sync attempted when one or more SpectroSERVERS are not connected to the OneClick server
Jan 16, 2019 15:15:54.965 (http-nio-80-exec-9) (NetqosSyncLog) - (ERROR) - Sync attempted when one or more SpectroSERVERS are not connected to the OneClick server

Cause

  • This can happen after upgrade to PM 3.x and spectrum 10.x.
  • This can also be seen in systems where a network problem causes the OC and SS systems to lose connection.
  • In a standalone situation, when the SpectroServer (SS) goes down, neither sync to CAPC nor event polling are able to run.
  • When the SS comes back up (without a tomcat restart), it should be marked as synchronized, and start event polling. During the next sync, it should sync anything changed since the SpectroServer went down. OneClick will ask for events back to when the SS went down.
  • When SpectroServer is configured with Fault Tolerance (FT), it should operate as above, however, this was not happening. No synchronization or sending events should occur while the system is in a failover state and running on the (secondary) SpectroServer.

Environment

Spectrum 10.3.x
PM 3.x
One Fault-Tolerant pair (1 primary and 1 secondary SpectroServer)

Resolution

Issue was replicated and fixed, verified in both standalone and DSS setups. After applying this fix, event polling will start automatically when the SS returns to active state without a OneClick server restart.

This fix is available in the Spectrum 10.3.2 release:

-------------------------
Symptom : Alarm sync from CA PM to Spectrum does not starts when primary SpectroServer comes up(restart).
Resolution: Alarms sync from CA PM to Spectrum starts when primary SpectroServer comes up(restart).
(DE402443, 01280050)
-------------------------

This will result in the sync not working while the Primary SpectroServer is down (no sync via secondary SS). But it should work again, once the primary SS is running again, syncing since SS stopped. Another workaround is also to "enable/disable" the PM integration on the OneClick web page.

Until the above fix is able to be applied a restart of the Spectrum One Click tomcat web server should resolve the problem.

Additional Information

For Spectrum 10.3.0, a PTF was created - "Spectrum_10.03.00.PTF_10.3.021":