Watch4net: How to backup the Watch4net environment

Products

• VMware

Environment

VMware Smart Assurance - Watch4Net/M&R

Resolution

How to backup the Watch4Net environment



This is a list of what to copy when doing a backup of your Watch4net environment.
Although there are many ways to perform a backup and many applications to do so, this article will provide tips and tricks on how to perform the best backup possible and to reduce maintenance downtime to a minimum.

Generally, you want files not to be changed while being backed up. Configuration files are not modified while Watch4net is running, as they are loaded in RAM. However, the databases, log files and temporary files are constantly modified. You will find below some indications on how to avoid these issues. More importantly, periodically test data restoration to make sure backup works better and prepare yourself when a disaster really occurs and you need to recover data quickly. Linux / Solaris

Databases: We recommend locking the databases while doing a backup rather than stopping the database engine. You can use the following scripts to lock the databases:

• /opt/APG/bin/lock-apg-database.sh    ----- locks only the Watch4net database
• /opt/APG/bin/lock-mysql-database.sh  ----- locks all databases 
• These scripts contain a timer of 2 hours (7200 seconds) by default, allowing you plenty of time to backup the databases using your favorite method. If you need to change the locking period, you can edit the scripts to set a timer that better suits your needs.
• You can also use another mechanism to lock the databases, like splitting the lock and unlock into a pre-backup and a post-backup scripts. If you opt for a pre-backup script, perform this command as root. FLUSH TABLES WITH READ LOCK; and keep session running while backup. Use post-backup scripts to end the "FLUSH TABLES WITH READ LOCK" session on pre-backup scripts. Only the same session can UNLOCK TABLES or session terminates will releases all table locks. Please refer to MYSQL documentation. Also you can use schedule task to manage this operation with execution-locks "database-maintenance". Please refer to "Task scheduler administration Guide" for more information.
• While the database is locked, collection will continue and data will be cached by the backend and pushed into the database when it's unlocked.
• You can copy the MyISAM files using your favorite archiving tool. The MyISAM files are located into:
  /opt/APG/Databases/MySQL/Default/data/apg     -- The main Watch4net database
  /opt/APG/Databases/MySQL/Default/data/master  -- Used for user authentication, and to store user's profile 
• We strongly discourage the use of MySQL dump (mysqldump command) on large database (>10GB). The mysqldump command can run for several hours (if not days) on large databases. The backup of database's raw files is a much better option with less overhead.
• Please contact EMC Watch4net Support Department if you have questions regarding backing up the database, or if you plan to use an advanced backup strategy, like incremental backup or dual server environment please speak with your EMC Account Manager regarding our Professional Services 

Datastore: Although Watch4net does not provide scripts to lock the Datastore databases (used for events and netflow collection), the logic is the same.

• Perform this command as root using datastore-client.sh:
• FLUSH TABLES WITH READ LOCK;
• Backup the database with your favorite archiving tool
• Run this command as root to unlock the database:
• UNLOCK TABLES;
• Datastore files are located under:
• /opt/APG/Databases/APG-Datastore/Default/data/events  ---- Used for storing events
• /opt/APG/Databases/APG-Datastore/Default/data/flow    ---- For Traffic Flow Data 
• We strongly advice not to use the mysqldump command to backup the datastore, as it may run for several hours, even days, and the dump won't be compressed like the Database is. Although your backup implantation allows for data compression, it is likely not as efficient as the Datastore. No need to spend time decompressing and recompressing data already compressed

Configuration files: Configuration files bear standard extensions. If you select per file extension, you will be able to get all configuration easily.

• Extensions are:
• .xml
• .properties
• .cnf
• .ini
• Other files are stored in the same folders as the configuration files. These are validation files (extensions .xsd and .dtd). You may optionnaly back them up. However, if your backup strategy is to install a fresh Watch4net then restore your data onto it, the validation files are not required.
• Make sure to keep the folder arborescence when archiving configuration files. Restoration will be a lot easier.

Folders to avoid: Some folders contain temporary data, whether log files or cached data. As these are files change often, it is advised not to back them up.

• These folders are:
• All "logs" folders
• /opt/APG/Collecting/FailOver-Filter/Default/tmp-backend  ---- When the Fail-Over Filter is used.
• /opt/APG/Backends/APG-Backend/Default/tmp                ---- Where the Backend caches its data.

Binary Files: Depending on your backup & restoration strategy, it may not be necessary to save scripts and binary files.

• If you plan to perform a fresh install of Watch4net then to restore your data and configuration files, no need to save the scripts and binaries.
• Important: Make sure you restore your data onto the very same version as the version used when the files were backed up.
• If you plan to restore your whole Watch4net installation from the backup media, then you need to backup all files and folders under:
• /opt/APG
• Stop all services using this command:
• /opt/APG/bin/manage-modules.sh service stop all
• Important: Collection will be stopped during this time, causing gaps in your reports.

Windows

The logic of Watch4net under Windows is similar to Watch4net under Linux. Thus, only specific locations will be listed. Please refer to the Linux / Solaris section for more details on backup implantation.

• Lock scripts: The scripts to lock the databases are located at:
• C:\Program Files\APG\bin\lock-apg-database.cmd    ----- locks only the Watch4net database
• C:\Program Files\APG\bin\lock-mysql-database.cmd  ----- locks all databases

 

• MyISAM Files Location: Copy the content of these folders:
• C:\Program Files\APG\Databases\MySQL\Default\data\apg    ---- Watch4net Main Database
• C:\Program Files\APG\Databases\MySQL\Default\data\master ---- Authentication Database 

 

•  Datastore Files Location:
• C:\Program Files\APG\Databases\APG-Datastore\Default\data\events  ---- Used for storing events
• C:\Program Files\APG\Databases\APG-Datastore\Default\data\flow    ---- For Traffic Flow Data

• Folders not to backup: Do not backup these folders as they contain temporary data.
• All "logs" folders
• C:\Program Files\APG\Backends\APG-Backend\Default\tmp        ---- Used for caching Backend's data
• C:\Program Files\APG\Collecting\FailOver-Filter\Default\tmp  ---- When the FailOver filter is used