Spectrum Archive Manager not starting after system crash.
search cancel

Spectrum Archive Manager not starting after system crash.

book

Article ID: 57378

calendar_today

Updated On:

Products

CA Spectrum DX NetOps

Issue/Introduction

Spectrum Archive Manager not starting after system crash. The following error message is seen in the $SPECROOT/SSS/DDM/ARCHMGR.OUT file:

"Native error: 145, table './ddmdb/event' is marked as crashed and should be repaired" 

Environment

Any version of Spectrum

Cause

The system crashed caused the Archive Manager DDM database to become corrupt.

Resolution

1. Run ArchMgr -repair to repair the database by doing the following:

  • Log into the system as the user that owns the Spectrum installation
  • If on Windows, open a bash shell by running "bash -login". If on Linux or Solaris, open a terminal window.
  • cd to the $SPECROOT/SS/DDM directory
  • Remove the .DDMDB.LOCK file if present and Archive Manager is not running
  • Run the following command: ./ArchMgr -repair


NOTE: Depending on the size of the DDM database, the repair could take a few minutes up to several hours (if DDM is many GB in size). If the repair is successful, start the Archive Manager with Spectrum Control Panel. To start Archive Manager via command line, see Additional Notes below. 

NOTE: In the following MySql commands, replace <PASSWD> with the root password for your DX NetOps Spectrum version.


2. If the above repair was not successful, you can try ./ArchMgr -force -repair

3. If still not successful, you may need to perform some additional maintenance on the MySQL database

  • cd to <SPECROOT>/mysql/bin
  • run ./mysqlcheck -uroot -p<PASSWD> ddmdb

 

If errors are found, mysqlcheck will attempt to aut-repair them. If for some reason there is issue with the repair, you can try to Force Repair: 

  • ./mysqlcheck -f -r -uroot -p<PASSWD> ddmdb


You can also try running repair on the DB files individually: 

  • run ​./myisamchk -m data/ddmdb/*.MYI
  • if errors are found, run ./myisamchk -r data/ddmdb/*.MYI


NOTE: Starting in Spectrum 10.3 we are no longer using MyISAM MySQL Engine and have now moved to newer more efficient MySQL InnoDB Engine, thus this command will only work on 10.2.3 and lower.

Lastly, there is an option to set Archive Manager to repair automatically. Be Aware that for larger databases, this can cause significant Archive Manager startup delays, so use with caution.

  • add the following line to <SPECTRUM>/SS/DDM/.configrc: AUTO_REPAIR_DB=TRUE
  • save and restart Archive Manager from Spectrum Control Panel

Additional Information

Archive Manager is down in OneClick Connection Status - validating database
https://knowledge.broadcom.com/external/article?articleId=32832

 

Start Archive Manager via command line: 

  • cd to <SPECROOT>/bin
  • ./launchinstdbapp localhost ARCHMGR y ARCHMGR.OUT

 

Stop Archive Manager via command line (This can be used to gracefully stop ArchiveManger for testing fault tolerance):

  • cd to <SPECROOT>/bin
  • ./cmdC localhost 2 ARCHMGR

 


If you also experienced a Spectrum SpectroSERVER crash and need instructions on how to load a backup database for recovery, please reference knowledge article 113595.

For more information on repairing a corrupt DDM database, please reference the "DDM and Archive Manager Troubleshooting" section of the documentation.

Powered by Wolken Software