Historical trace information not available
Article ID: 33220
Updated On:
Products
Issue/Introduction
Problem:
Historical transaction trace data is not available either in WebView or Workstation. The header information is available but the details will not be displayed
and you will see an error as follows:
<Please see attached file for image>
This is generated in one of two ways:
a) by putting the Investigator into historical mode then selecting a time frame before today and selecting the Traces or Errors tab and choosing a particular event,
b) or by using the Historical Query Viewer to search for any type of event and selecting a time frame for a day before the current one.
Symptoms
- Today's traces will display with no problem.
- If you take a copy of the traces folder and review with the Lucene tool, all trace information will be available.
- If you take the trace information to another Enterprise Manager, you may not get the same problem.
- You would also notice this error in the Enterprise Manager log in VERBOSE logging
[VERBOSE] [PO:main Mailman 3] [Manager.TransactionTracer] Failed to get historical trace: 1433938415115:8081
com.wily.introscope.server.enterprise.entity.transactiontransactiontrace.storage.HistoricalPerstStorage.open(HistoricalPerstStorage.java:213)
at com.wily.introscope.server.enterprise.entity.transactiontrace.storage.MultiStorePerstStorage.openHistoricalStorageIfNecessary(MultiStorePerstStorage.java:728)
at com.wily.introscope.server.enterprise.entity.transactiontrace.storage.MultiStorePerstStoragetrace.storage.TransactionTraceStorageException: Unable to open the Perst backing store: /opt/apm/traces/traces_20150610.db
at com.wily.introscope.server.enterprise.entity..getTraceFromHistoricalStorage(MultiStorePerstStorage.java:716)
at com.wily.introscope.server.enterprise.entity.transactiontrace.storage.MultiStorePerstStorage.doGetHistoricalTrace(MultiStorePerstStorage.java:592)
at com.wily.introscope.server.enterprise.entity.transactiontrace.storage.MultiStorePerstStorage.getHistoricalTrace(MultiStorePerstStorage.java:353)
at com.wily.introscope.server.enterprise.entity.transactiontrace.storage.TransactionTraceStorageSystem.getHistoricalTrace[..]
Caused by: org.garret.perst.StorageError: Failed to locate org.garret.perst.impl.AltBtree$BtreePageOfString: java.lang.ClassNotFoundException: org.garret.perst.impl.AltBtree$BtreePageOfString (com.wily.isengard.tunnelingservice)
at org.garret.perst.impl.ClassDescriptor.loadClass(ClassDescriptor.java:353)
at org.garret.perst.impl.ClassDescriptor.onLoad(ClassDescriptor.java:358)
[..]
Note the java.lang.ClassNotFoundException for org.garret.perst.impl.AltBtree$BtreePageOfString with mention of com.wily.isengard.tunnelingservice
as a key indicator.
Environment
The problem was discovered in APM 9.7.1 in a standalone Enterprise Manager running on Linux. However, this has been reproduced with Enterprise Manager running on Windows.
We have seen this error specifically where .NET agents are connecting to the HTTPS port of the Enterprise Manager. Connecting Java agents to the HTTPS port
did not bring about the error.
Cause
The Enterprise Manager is an OSGi application which means it is built up of many bundles of jar files, each with their own classloader.
Access is restricted only to specifically allowed classes as a basic principle of OSGi framework.
Due to changes in the com.wily.isengard.com.wily.isengard.tunnelingservice_x.x.x.jar since APM 9.7, we had to add an exception to allow access
to the org.garret.perst classes, available in the com.wily.perst_2.6.4.v20070914.jar bundle.
Resolution
To allow access to the org.garret.perst classes, do the following:
1) Stop the Enterprise Manager
2) Go to the Enterprise Manager's /product/enterprisemanager/configuration folder
3) Delete all files from the folder except the config.ini file
4) Add the following line to the config.ini file:
org.osgi.framework.bootdelegation=org.garret.perst.*
5) Restart the Enterprise Manager
Environment
Component:
Attachments
Feedback
