Troubleshooting Spectrum Failed OLB, hung, paused, Online Backup Model alarm
search cancel

Troubleshooting Spectrum Failed OLB, hung, paused, Online Backup Model alarm


Article ID: 202079


Updated On:


CA Spectrum DX NetOps


Troubleshooting Spectrum Online Backup


Release: All Spectrum Versions

Component: SPCCSS - SpectroSERVER Core


Check Online Backup model for events and alarms: 

1. In OneClick, go to Locator Search
2. search By Application Models > By Name
3. search "online" to locate all Online Backup models in your DSS environment
4. select the OLB model having issue and review events and alarms

The Spectrum Online Backup fails with the following two Events/Alarms on the OnLineBackup model:

OnLineBackup (name - OnLineBackup) has reported a general failure!.

If an Online Backup has failed previously, the tmp file created may be left, preventing a new backup attempt. 

1. Check for a left over "tmpdir" directory in the location where the backup files are saved during an Online Backup process.
2. Remove any left over "tmpdir" directories and try another backup

Hanging Online Backup: 

  1. Delete the temp directory for the backup from $SPECROOT/SS-DB-Backup
  2. Go to the OnLineBackup model in OneClick
  3. Go to the Attributes tab
  4. Change the following:
    1. BackupNow to "No" 
    2. BackupStatus to "Ready" 
    3. PauseVNM to Yes 

You should now see "Begin Backup Now" and "Ready" under the VNM model > Online Database Backup. This should allow you to run an OnlineBackup. If it hangs again please open a support ticket.

Online Backup Fails with Error : Failed To Open Destination File:

There are several reasons why this could occur:

  • When there is a temporary directory (*tmpdir) left over from a previous backup in the $SPECROOT/SS-DB-Backup directory (default), or the directory configured by the user to save the backup to. 
  • When there is not enough space available to save the file(s).
  • Permissions on the backup directory have been changed.
  • On Windows, if there is another rm.exe program installed on the system besides the one in the $SPECROOT/NT-Tools/SRE/bin directory

To solve: 

  • Check the contents of the backup directory for any left over tmpdir directory and remove
  • Ensure there is available space
  • Check and verify the permission on the backup directory
  • Remove any rm.exe programs other than the one in the $SPECROOT/NT-Tools/SRE/bin directory


Enable Online Backup Debugging: 

1. Within OneClick, go to VNM model of the same landscape where the Online Backup issue exists
2. Go to Information > Dynamic Debugging
3. Enable Online Backup debug

The debug outputs to <SPECROOT>/SS/VNM.OUT

You can also enable debug via command line


Fault Tolerant Database Sync Failure: 

Review the following log files for errors. 

on Primary server:


on Secondary server: 


Database Sync occurs on port 51966 - 0xCAFE - by default. Make sure this is open in local and network firewalls, or change in /SS/.vnmrc: rcpd_comm_port=<port number in hex>


Additional Information

There may be times when the SpectroSERVER threads are consumed causing latency issues. This may prevent the online backup from running after changing the attributes.  If you try to stop the SpectroSERVER, you are unable to stop it.  In this instance you will need to kill the SpectroSERVER and reload a previously saved copy from the last time it successfully ran.


Hanging online backup

Fault Tolerant Backups take a long time