Spectrum Archive Manager not starting after system crash.
Article ID: 57378
Updated On:
Products
Issue/Introduction
Spectrum Archive Manager not starting after system crash. The following error message is seen in the $SPECROOT/SSS/DDM/ARCHMGR.OUT file:
"Native error: 145, table './ddmdb/event' is marked as crashed and should be repaired"
Environment
Any version of Spectrum
Cause
The system crashed caused the Archive Manager DDM database to become corrupt.
Resolution
1. Run ArchMgr -repair to repair the database by doing the following:
- Log into the system as the user that owns the Spectrum installation
- If on Windows, open a bash shell by running "bash -login". If on Linux or Solaris, open a terminal window.
- cd to the $SPECROOT/SS/DDM directory
- Remove the .DDMDB.LOCK file if present and Archive Manager is not running
- Run the following command: ./ArchMgr -repair
NOTE: Depending on the size of the DDM database, the repair could take a few minutes up to several hours (if DDM is many GB in size). If the repair is successful, start the Archive Manager with Spectrum Control Panel. To start Archive Manager via command line, see Additional Notes below.
NOTE: In the following MySql commands, replace <PASSWD> with the root password for your DX NetOps Spectrum version.
2. If the above repair was not successful, you can try ./ArchMgr -force -repair
3. If still not successful, you may need to perform some additional maintenance on the MySQL database
- cd to <SPECROOT>/mysql/bin
- run ./mysqlcheck -uroot -p<PASSWD> ddmdb
If errors are found, mysqlcheck will attempt to aut-repair them. If for some reason there is issue with the repair, you can try to Force Repair:
- ./mysqlcheck -f -r -uroot -p<PASSWD> ddmdb
You can also try running repair on the DB files individually:
- run ./myisamchk -m data/ddmdb/*.MYI
- if errors are found, run ./myisamchk -r data/ddmdb/*.MYI
NOTE: Starting in Spectrum 10.3 we are no longer using MyISAM MySQL Engine and have now moved to newer more efficient MySQL InnoDB Engine, thus this command will only work on 10.2.3 and lower.
Lastly, there is an option to set Archive Manager to repair automatically. Be Aware that for larger databases, this can cause significant Archive Manager startup delays, so use with caution.
- add the following line to <SPECTRUM>/SS/DDM/.configrc: AUTO_REPAIR_DB=TRUE
- save and restart Archive Manager from Spectrum Control Panel
Additional Information
Archive Manager is down in OneClick Connection Status - validating database
https://knowledge.broadcom.com/external/article?articleId=32832
Start Archive Manager via command line:
- cd to <SPECROOT>/bin
- ./launchinstdbapp localhost ARCHMGR y ARCHMGR.OUT
Stop Archive Manager via command line (This can be used to gracefully stop ArchiveManger for testing fault tolerance):
- cd to <SPECROOT>/bin
- ./cmdC localhost 2 ARCHMGR
If you also experienced a Spectrum SpectroSERVER crash and need instructions on how to load a backup database for recovery, please reference knowledge article 113595.
For more information on repairing a corrupt DDM database, please reference the "DDM and Archive Manager Troubleshooting" section of the documentation.
Feedback
