Reporter reports high memory for over months
Article ID: 281952
Updated On:
Products
Issue/Introduction
Reporter reports high memory for over months. Please see the snippet below, and advise. accordingly.
Cause
From what's been shared the memory is still in the "OK" state, and does not represent a problem. Out of abundance of caution, you may want to do a purge on the database, to have the usage come down.
Resolution
Each database creates and manages its own memory-resident log table. Each log table is composed of hour-tables containing data for each hour the database log processors spend reading log files. These tables typically consume most of the active memory in Reporter and therefore have a significant impact on overall log-processing performance. (When you unload a database it no longer consumes active memory.)
If all log files were processed in chronological order, more than one hour-table would not be necessary in memory. Commonly, the log-processing process encounters batches of log files that span multiple hours between them. If they are processed out of chronological order, performance significantly improves by allowing the number of hour-tables to grow, provided there is sufficient process memory. Conversely, during low memory conditions, reducing the number of hour-tables prevents unnecessary memory starvation and subsequent disk operations (swapping files in and out of memory).
Reporter orders log files using a numeric field in the filename, when it is present. The default filenames that are created by the ProxySG contain a Month/Day/Hour/Minute/Second timestamp immediately preceding the .log or .log.gz suffix; for example: SG_Main_HQ-1_1102081500.log.gz. If the filename ends with .log or .log.gz, the log processor parses it for any purely numeric sequence immediately preceding the required suffix. If one is found, it is then used to sequentially order that batch of log files. You can significantly improve log processor performance by naming the log files with any ordered numeric values that comply with this format. For example: anyfilenameprefix123.log or some-other-prefix-84757.log.gz.
Howbeit, it's important to note that most of the database is kept in memory. If the entire database is not occasionally purged, it continues to consume more of the process memory as new log files are processed. As the database grows, configuration settings that were previously beneficial might become detrimental.
As a general guideline, Symantec recommends that databases contain a maximum of 30 days of log data. However, the amount of log data (number of rows) has more impact than the number of days (age of data) in the data sets.
Reporter also allows the administrator to purge the database based on the number of log lines. Purge the log lines by expiration, automatically (scheduled), or manually.
After you create databases and begin generating and managing reports (filters, emailing, and so on), you might find a need to modify existing configurations.
You can configure Reporter to purge databases or logs that are based on a schedule or disk capacity threshold.
For example, please see scenarios 1 & 2, in the Tech. Doc. with the URL below. Execute the scenario that best applies to your implementation and environment.
Feedback
