UIM - QOS Data Retention
search cancel

UIM - QOS Data Retention


Article ID: 137764


Updated On:


DX Unified Infrastructure Management (Nimsoft / UIM)


We have a business need to retain metric data for systems for an extended period of time, e.g., ~1 year.  Are there any mechanisms available for archiving performance data within UIM event if the system were deleted to avoid monitoring alerts?


Release : 9.1.0

Component : UIM - DATA_ENGINE




Nimsoft Alarm Server (nas) - alarms


If you need to store alarms for a longer period of time, its simple and easy. Alarms are stored in 3 backend tables, nas_alarms, nas_transaction_summary and the nas_transaction_log tables. The nas contains transaction log retention settings that can be modified by the UIM administrator as needed. Reporting tools can be set to access this data where it sits or offloaded externally or to a file, to be used by reporting tools.


data_engine (QOS)


In the UIM data_engine probe there are data management/maintenance functions.


* RN tables are for the raw QOS data

Most large customers set this to ~30 days to avoid tables growing too large


* HN tables are for the hourly historical data and will start populating after the raw data days run out.

Most large customers set this to 180. Using the two settings I just mentioned you would hold up to ~210 days worth of data.


* DN tables are for the daily historical data and will start populating after the historical data days run out


During scheduled maintenance runs, the data_engine probe compresses and aggregates raw data from the RN tables into hourly data that is stored in the historic data (HN) tables. 


Granted, we know that some companies, due to the need for management reporting, or due to audits, regulations/HIPAA, or other business needs, e.g., the need to store data for longer periods of time, but if your tables become too large by saving too much raw/historical data, then data maintenance may slow down normal operations or portal performance if the tables grow too large. Table growth depends on several factors such as how many metrics are being monitored and how often (frequency). Best Practices for the UIM database assume a DBA/DBA Group is managing the UIM backend database. It also assumes that partitioning is enabled.


So ideally, 30 and 180 are normally the optimal raw/historical settings but if you need to store more data for a longer period of time, and its mainly for reporting, a DB archival strategy (Data archiving is the practice of moving data that's no longer being used to a separate storage device) can be implemented for UIM, but there is currently no formal built-in archival function to 'offload' data from the UIM database for later access and reporting.


Normally a solid DBA or the DBA Group will possess some collective knowledge and/or preferred approaches for archiving strategies, most often for performance or audit reporting. Some helpful information can be found online for the given database vendor/type, e.g., MS SQL Server,   Oracle, or MySQL