- Upgrade to the latest version of Reporter
- Use the sizing guide found here. (Sizing for Reporter OS alone and not to include FTP logs File Storage)
- Only send access log files from the ProxySG in the optimized format: bcreportermain_X.
- Use only the recommended browsers, the list of recommended browsers can be found in the release notes for the version of Reporter you are running.
- Ensure only one type of logs are sent to each database; each of the below access logs should be in separate databases.
- Main (HTTP)
- SSL (HTTPS)
- Streaming (Streaming media logs)
- Proxy Client logs
- Optimizing log processing is discussed in Chapter 2 of the Reporter Administration Guide, under the heading "About Optimizing Log Processing Configurations" follow these recommendations to optimize the processing of logs.
NOTE: Version 10.x Reporter supports Broadcom ProxySG HTTP, HTTPS, Proxy Client access logs and Content Analysis.
- Use FTP to upload your reports to a local drive on the Reporter server.
- Ensure you always test how long a Report takes to run before you schedule it. This ensures that you allow appropriate time between scheduling each report.
- Do not store more than 50,000 access logs in one folder. Reporter has trouble with more than this amount, when it comes to having to re-process them all at once.
- Schedule Reporter to process it's access logs at 2 AM every day to avoid running reports, and processing access logs at the same time.
- Do not upload more than a year worth of access logs to the same folder. By default, the ProxySG does not write the year into the name of access log, so after a year, Reporter sees the same name log file in the folder. The rename occurs as Reporter moves or rename the access logs once it has finished processing them. The file names will look like the following:
Original file name from the year before: SG_main__2100316010000.log.gz
After an attempt to rename/move the access log file, it will name this file to: dup0001_SG_main__2100316010000.log.gz
Shut down gracefully:
- Ensure Reporter is shutdown gracefully before stopping the Host operating system.
- For LINUX, run the "/etc/init.d/bcreporter stop" command before stopping LINUX.
- Before scheduling a report, manually generate it to time how long it takes.
- Schedule reports at different times.
- Predict how long a report will run and schedule large reports at staggered intervals from each other.
- Schedule large reports at the slower times of the day, such as after midnight.
- To allow for future growth, install your application on VMware. This allows you to add memory, as your database grows.