Best practices to move Spectrum to a new system
search cancel

Best practices to move Spectrum to a new system

book

Article ID: 128599

calendar_today

Updated On:

Products

Spectrum Network Observability

Issue/Introduction

This document outlines the steps needed to migrate or move your Spectrum installation from one system to another.

A common example of this is when the environment requires OS upgrades. These will often entail standing up new physical or virtual systems with new resources to support the OS upgrade.

How to migrate SpectroSERVER servers when the OS is upgraded?

How to migrate the OneClick servers when the OS is upgraded?

Environment

All supported DX NetOps Spectrum releases

Cause

Environmental requirements for OS upgrades on a regular basis.

Resolution

This covers a scenario where we are tied to OS upgrades requiring new systems. OS in-place upgrades are not possible in this scenario.

  • Migrating to new servers without upgrading the NetOps software version
  • Involves a DSS Spectrum environment with:
    • 1 or more SpectroSERVERs in a Primary/Secondary Fault-Tolerant Pair
    • 1 or more OneClick servers

NOTES:

  • Source system refers to the original existing system.
  • Destination system refers to the new system being built.
  • SS and OC systems can be moved in parallel to save time.
  • When installing Spectrum on the destination systems ensure the matching landscape type is set. If source is using huge or legacy, ensure the destination is installed using the same configuration.

Complete these steps.

  1. Prepare the new servers with the new OS as required by the environment.
  2. Ensure the new servers meet all System Requirements and Prerequisites
  3. Integrated with DX NetOps Portal?
    1. Recommended that the integration be disabled during the migration process.
    2. Enable it again when the migration is completed.
    3. To Enable/Disable the Spectrum Data Source:
      1. Log into the Portal UI.
      2. Go to Administration->Data Sources->Data Sources.
      3. Choose the Spectrum Data Source and set it to the Enabled or Disabled state as needed.
  4. Migrate the Primary SS
    1. Backup all necessary data from the Primary SS source system and shut down the server.
      • The Secondary FT SS should take over.
    2. New servers will be set to use the same name and IP as the old hosts? Before installing Spectrum:
      • Ensure the source system is shut down.
      • Change host name and IP on destination system to use the name and IP the source system used.
      • Validate DNS and IP resolution works properly before continuing.
    3. Install Spectrum on the Primary destination SS system. After installing Spectrum ensure the services are shut down and:
      1. If .hostrc as well as etc/hosts file changes are required in the source system reproduce them in the destination system.
      2. Load the SSdb and Archive Manager DB.
        1. Load the SSdb with the SSdbload command.
          1. As the spectrum user
          2. Uncompress the backup if compressed (.gz, gzip -d <backupName>.SSdb.gz)
          3. Go to the $SPECROOT/SS/ directory and run:
            • ../SS-Tools/SSdbload -il <path_to_backup>.SSdb 
        2. Load the ArchMgr DB with the ddm_load command
          1. As the spectrum user
          2. Uncompress the backup if compressed with .gz extension.  If the extension is of type .mbi then do not uncompress it.  This is a mysql extension.
          3. Navigate to the $SPECROOT/SS/DDM directory and run:
            • ./ddm_load.pl -f <filename.mbi>
      3. Start the SSdb.
  5. Migrate the Secondary SS.
    1. Backup the source Secondary and shut it down.
      • Confirm new Primary SS takes over.
    2. New servers will be set to use the same name and IP as the old hosts? Before installing Spectrum:
      • Ensure the source system is shut down.
      • Change host name and IP on destination system to use the name and IP the source system used.
      • Validate DNS and IP resolution works properly before continuing.
    3. Install Spectrum on the Secondary destination SS system. After installing Spectrum ensure the services are shut down and:
      1. If .hostrc as well as etc/hosts file changes are required in the source system reproduce them in the destination system.
      2. Run OLB to get SSdb from new Primary backed up and loaded on new destination Secondary.
      3. Start the SSdb.
      4. If a DDMDB was migrated on the Primary load it on the Secondary.
        1. Ensure processd, the SS and other related services are up.
        2. Ensure Archive Manager is down. It must be down, with MySql running, when the DDMDB is loaded.
        3. Load the ArchMgr DB with the ddm_load command
          1. As the spectrum user
          2. Uncompress the backup if compressed.  If the extension is of type .mbi then do not uncompress it.  This is a mysql extension.
          3. Go to the $SPECROOT/SS/DDM directory and run:
            • ./ddm_load.pl -f <filename.mbi>
        4. Start the Archive Manager.
    4. Validate Secondary shows properly in the OneClick Administration site Landscapes page.
  6. Migrate the OneClick server
    • Backup OC system and shut it down.
    • Change host name and IP on OC destination system.
      • Validate DNS and IP resolution works properly before continuing.
    • Install Spectrum OC on destination system.
      • Shut down tomcat and webtomcat services.
      • Confirm they are down.
      • Restore the backed up files from source to destination
      • Start the tomcat and webtomcat services.

 

Backup steps for SS

    1. SS Database (SSdb) backup
      1. Create a current backup using SSdbsave or the OLB backup in OC.
      2. Transfer the $SPECROOT/SS-DB-Backup folder to the destination system.
    2. Archive Manager DDMDB backup.
      • Generate the backup using:
        1. cd $SPECROOT/SS/DDM/
        2. ./ddm_save.pl -f ddmdb_date
    3. Alarm Notifier applications running on the SS?
      • Copy and move the Alarm Notifier folders/files from source to destination system
      • Ensure any required customized $SPECROOT/lib/SDPM/partslist/*.idb files are copied from source to destination system.
    4. Back up/save configuration files that may be needed on the new system such as the custom folder, and Notifier folder.  Run the getSpectrumInfo.sh script on the original system:
      1. cd $SPECROOT/
      2. ./bin/support/getSpectrumInfo.sh
      3. This will create a file in $SPECROOT named logs-<hostname>-<date>.tar.gz

Backup steps for OC

      1. Back up/save configuration files that may be needed on the new system such as the custom folder, tomcat and webtomcat configuration files.  Run the getSpectrumInfo.sh script on the original system:
        1. cd $SPECROOT
        2. ./bin/support/getSpectrumInfo.sh
        3. This will create a file in $SPECROOT named logs-<hostname>-<date>.tar.gz
    2. OneClick MySql database backups.
      • If your OneClick host serves as an SRM host, or as a NetOps Portal integration host, backup their databases.
        • For SRM be sure to backup the DB named reporting.
        • For the NetOps Portal integration be sure to backup the DB named netqos_integration.
        • This documentation covers how to back up those two databases.
      • If you have custom MIBs imported to the OC database follow the steps in the following KB article to backup and transfer it.
      • Load a MySQL database into the new OC host, shown using the netqos_integ database as an example.
        1. Move a copy of the backed up netqos_integ_backup.sql to the destination OC system.
        2. Change directories:
          • cd $SPECROOT/mysql/bin/
        3. Run:
          • ./mysql --defaults-file=../my-spectrum.cnf -uroot -p<password> netqos_integ < /<PathToFile>/netqos_integ_backup.sql
            • Replace <password> with the correct MySql DB password.
        4. NOTE: The same steps can be used for the reporting DB, just replace the DB name and files in the commands.
        •  

Backup steps for the SpectrumDataPublisher

        • Backup and transfer the following files to the destination system.
          • <SDP_DIR>/config/ConnectorConfig.xml
          • Security file: <SDP_DIR>/secure.properties
          • Keystore for OneClick and OI certs: <SDP_DIR>/Security/

             

Additional Information

  • Multiple SS systems in your DSS?
    • Migrate the MLS SS first.
    • Continue with Primary, then Secondary pairs, one Fault-Tolerant pair at a time until completed.
  • Multiple OC systems in your environment?
    • Migrate at least one of the OC servers before the first Secondary SS is migrated.
    • After this migrate the remaining Primary/Secondary SS pairs.
    • Once all remaining Primary/Secondary SS pairs are migrated, migrate the remaining OC servers.

On your new OneClick if you had SSO and/or LDAP configured you may need to reconfigure it.

 

There may be instances in which the locserver map does not properly update with the primary and secondary host information.  To resolve that you need to save the SSdb on the primary, load it on the secondary, save it on the secondary and reload it on the primary.  You can ignore the install comments in the steps below if you have already installed:

Made sure to keep the SS down in the new env and ran through this:

 Save the SSdb on the original production boxes

Move it to the new box - unzip it, rename it to dbsavefile.SSdb and put it in the $SPECROOT/SS directory

Run the install and once complete do not start the SS

mkdir SS-DB-Backup on the primary

Manually save the SSdb from the SS directory:

../SS-Tools/SSdbsave -cm ../SS-DB-Backup/db_fromprimary.SSdb

Install on the secondary

mkdir SS-DB-Backup on the secondary

Move that file to the secondary and load it:

../SS-Tools/SSdbload -ilr 20 ../SS-Tools/db_fromprimary.SSdb

Now save that SSdb on the secondary:

../SS-Tools/SSdbsave -cm ../SS-DB-Backup/db_fromsecondary.SSdb

Move that SSdb to the primary and load it:

../SS-Tools/SSdbload -il ../SS-DB-Backup/db_fromsecondary.SSdb

Start the primary - once it's 100% active, start the secondary SS

Powered by Wolken Software