SpectroSERVER Port 0x0 or Incorrect Precedence after SSdbload: How to Set Up Fault Tolerance

Products

• Network Observability
• Spectrum

Issue/Introduction

Fault Tolerance or high availability configuration is a redundancy configuration consisting of a Primary and a Secondary SpectroSERVER. This configuration ensures that monitoring is not interrupted if the Primary SpectroSERVER goes down.

 

Symptoms

• Secondary SpectroSERVER shows Precedence as 10 instead of 20.
• SpectroSERVER Port appears as 0x0 after a database load.
• Database files are incorrectly extracted into the $SPECROOT/SS-Tools/ directory.

Environment

• Product: DX NetOps Spectrum
• Version: All Supported Versions
• OS: Linux / Windows

Cause

This issue occurs if the SSdbload utility is executed from the $SPECROOT/SS-Tools/ directory. Even though the binary resides in SS-Tools, it must be executed from the $SPECROOT/SS/ directory to correctly reference the database environment. Additionally, running the command as the root user instead of the Spectrum Install Owner (e.g., ####admin) can cause permission discrepancies.

Resolution

To correctly configure the Secondary SpectroSERVER, follow these steps:

1. Perform a database save on the Primary server using either Online Backup or the SSdbsave command line utility.
2. Transfer the saved file to the Secondary machine using a secure transfer method (ensure binary mode if using FTP). Place the file in the $SPECROOT/SS/ directory.
3. On the secondary machine, stop the SpectroSERVER:
   • Run: $SPECROOT/bin/stopSS.pl
4. Navigate to the SS directory:
   • Run: cd $SPECROOT/SS/
5. Execute the SSdbload command using the Spectrum Install Owner account:
   • Run: ../SS-Tools/SSdbload -il -add 20 <savefile_name>
6. If SSdbload was previously run from the incorrect directory, delete the following stale files from $SPECROOT/SS-Tools/:
   • *.db
   • *.ix
   • SSdbload.log
7. Start the SpectroSERVER on the Secondary machine.
8. Schedule daily Online Backups on the Primary server to maintain synchronization.

Additional Information

• Landscape handles must be unique in a distributed environment.
• Precedence values must be unique in a fault-tolerant environment (typically 10 for Primary, 20 for Secondary).
• For hot stand-by polling, add secondary_polling=yes to the .vnmrc file.
• Review established Fault Tolerance documentation