When does a CA Spectrum Fault Tolerant (Secondary) SpectroSERVER take over when the Primary goes down?

Products

• Spectrum

Issue/Introduction

In a CA Spectrum Fault Tolerant environment, when does a secondary SpectroSERVER know to become the active server when the primary goes down or there is a network problem preventing access to the primary SpectroSERVER?

Resolution

There are two parts to SpectroSERVER fault-tolerance, and each are independent from each other.

The secondary SS "polls" (via API call) the primary SS every minute.  If the polling determines the primary is down, the secondary starts SNMP polling, processing SNMP traps, etc, and essentially takes over the role of the primary SS.  If a network issue is preventing the polling, both SpectroSERVER's can be 'active'.

The Oneclick Server polls both the primary and the backup SpectroSERVER, via CORBA, every 10s and a 'deeper' poll every 60s.  If the polling determines the primary is down, OneClick will failover and connect to the secondary.  It is possible, if a network issue is blocking the communication, that OneClick server will failover to a secondary that is not 'active' which will display as a grey/suppressed VNM icon.
The API calls are not just "Are you up?" but require an intelligent response that only a non hung process could give.  This is to avoid problems when SS hangs but does not crash, and if we only checked the SS process was we would not fail over. So the primary need to give an intelligent response that only an active Ss process could.

If the primary becomes hung and doesn’t respond within 3 minutes, or there is a network problem preventing the request from reaching the primary (or the response from coming back) the secondary will become active.  
The 3 minute interval is configurable by setting the “vnm_message_timeout” (specified in milliseconds) in the .vnmrc file of the secondary.  CA recommends that you do not change this setting unless there are problems or you are instructed to change this by CA Support.