We are seeing errors in our Tomcat Agent logs.
[ERROR] [IntroscopeAgent.Agent] IntervalHeartbeat.execute threw executing: Waiting for Database Dynamic
java.lang.NoClassDefFoundError: java/sql/SQLException
at com.wily.introscope.agent.db.DatabaseManager$3.ITimestampedRunnable_execute(DatabaseManager.java:122)
at com.wily.util.heartbeat.IntervalHeartbeat$BehaviorNode.execute(IntervalHeartbeat.java:972)
at com.wily.util.heartbeat.IntervalHeartbeat.executeNextBehaviorAndCalculateSleepTime(IntervalHeartbeat.java:491)
at com.wily.util.heartbeat.IntervalHeartbeat.access$2(IntervalHeartbeat.java:445)
at com.wily.util.heartbeat.IntervalHeartbeat$HeartbeatRunnable.run(IntervalHeartbeat.java:668)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassNotFoundException: java.sql.SQLException
at com.wily.util.extension.JarExtension$AllPermissionsClassLoader.findClass(JarExtension.java:309)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at com.wily.util.extension.EagerAllPermissionsClassLoader.loadClass(EagerAllPermissionsClassLoader.java:115)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
This is a known issue. It is directly due to using Java 11 against an APM 10.7 Agent
Release : 10.7.0
Component :
As I promised , I would write up my findings and then close the case.
1) You are using APM 10.7 SP3 (old release) and Java 11. This is not supported. It may work. But that could be accidental and should not be relied on.
2) You are using NFS instead of SAN or some other Shared File Service. We don't recommend this for EM or Agents. I think the bold area may be contributing to your problem.
SmartStor on Network File Systems
Do not use NFS for SmartStor storage or replace SAN with NFS for SmartStor storage. NFS is optimized for file-sharing, not throughput. NFS is unacceptable for SmartStor because NFS automatically defers writes to make most efficient use of the I/O capacity. This deferred action causes major synchronization problem
3) The logs has this error
[ERROR] [IntroscopeAgent.Agent] IntervalHeartbeat.execute threw executing: Waiting for Database Dynamic
java.lang.NoClassDefFoundError: java/sql/SQLException
at com.wily.introscope.agent.db.DatabaseManager$3.ITimestampedRunnable_execute(DatabaseManager.java:122)
at com.wily.util.heartbeat.IntervalHeartbeat$BehaviorNode.execute(IntervalHeartbeat.java:972)
at com.wily.util.heartbeat.IntervalHeartbeat.executeNextBehaviorAndCalculateSleepTime(IntervalHeartbeat.java:491)
at com.wily.util.heartbeat.IntervalHeartbeat.access$2(IntervalHeartbeat.java:445)
at com.wily.util.heartbeat.IntervalHeartbeat$HeartbeatRunnable.run(IntervalHeartbeat.java:668)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassNotFoundException: java.sql.SQLException
at com.wily.util.extension.JarExtension$AllPermissionsClassLoader.findClass(JarExtension.java:309)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at com.wily.util.extension.EagerAllPermissionsClassLoader.loadClass(EagerAllPermissionsClassLoader.java:115)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
This is a known issue. It is directly due to using Java 11 against an APM 10.7 Agent:
Existing KBs
https://knowledge.broadcom.com/external/article?articleId=130737
https://knowledge.broadcom.com/external/article?articleId=226510
Your options are
- Use a 10.7 Agent with Java 8
- Use a SAAS Agent with Java 11 or later.
Thanks