Enterprise Manager data processing has to complete in under 7.5 seconds otherwise there will be issues with the way data is displayed and stored.
If either or both the Harvest and Smartstor Duration are taking too long, you may start to see gaps in data where the Enterprise Manager delays data processing to keep up.
The whole Harvest and Smartstor process involves the collecting of data from agents and calculators, updating alert statuses, returning query data and then the writing of that data to disk.
The Enterprise Manager waits for the writing of data to be completed before harvesting again. This is why the two are important to each other.
One way that the Smartstor Duration can be affected is if there are other concurrent disk operations, for example writing transaction trace data to disk.
To see if you are affected by this particular issue, review the perflog.txt.
An easy way to review perflog data, is to change the file extension to .csv and view in a spreadsheet viewer.
The columns that interest us are titled
Note this sample perflog data from an actual Enterprise Manager collector
<Please see attached file for image>
The Smartstor Duration column has values of 466,11326,6013,8470,407,5067,4454,241
The Trace Duration column has values of 44,9693,4219,4730,75,4405,3871,61
We can see spikes in Smartstor Duration at the same points as Trace Duration.
The writing of trace data to disk is not part of the Enterprise Manager's harvest and Smartstor process, so this can only be an example of disk contention.
Once a dedicated disk is specified for Smartstor, you must tell the EM about it by setting the "dedicatedcontroller" property to true:
If this property is set to false while a separate disk is used for Smartstor, some minor improvement will be achieved, but it is the combination of these three properties used together (and correctly) that inform the EM to utilize its optimization methods to best advantage.
This means that if you specify a disk located on an independent I/O controller, always set the "dedicatedcontroller" property to true, while, If you specify a disk which is not on an independent I/O controller, always set the "dedicatedcontroller" property to false.
Bear in mind that all sizing recommendations that we list are halved where a dedicated controller is not used.
Traces are started:
By default, 1 trace is taken from each agent every 2 minutes. Consider extending the sampling interval to every 30 minutes.
The properties in IntroscopeEnterpriseManager.properties must be uncommented to override the values on the agent:
You can set the sampling.perinterval.count to 0 to stop sampling if you wish.
Check in the CEM UI under Setup > Introscope Settings for the Transaction Trace settings. If you are seeing this problem, try to configure a short session for the trace, 5 minutes at maximum. As for the trace threshold, you have to consider the link between whatever threshold you have set for Slow Time defects and the percentage you set here.
If the Slow Time threshold is set at 5 seconds and you have a transaction trace time threshold of 20%, you will track transactions longer than 1 second in duration. Try to not set the value too low or you will have a flood of transaction traces.
You should also review how long you need to keep trace data, By default this is set to 14 days, but reducing to 7 days in IntroscopeEnterpriseManager.properties has helped in this situation: