The IBM Class Sharing feature, which is enabled out of box, causes classes to be stored locally. The classes could contain bytecode instrumented by the Agent. So even if the Agent is removed, those references are still loaded and can cause startup issues. This article addresses how to resolve this issue and what can be done to configure Class Sharing properly.
On one instrumented server, WebSphere does not start, even after uninstalling the Introscope Agent. Additionally, a different node in the cluster on the same machine does not start, and it is not instrumented.
When the first server is started and then trying to start the second server, the following error message appears, even though it is not configured for Introscope and it throws a "class not found" error in the logs. This "class not found" error message refers to introscope classes. For example:
SRVE0232E: Internal Server Error.
Exception Message: [com.wily.introscope.agent.trace.IMethodTracer]
This issue is the result of the Class Sharing feature implemented in the IBM 1.5 JVM or higher JRE's. (Even in 1.4.2 if J9 VM.)
The issue manifests itself in the following ways:
The issue has been resolved by:
However, the root cause appears to be that upon startup, by default the IBM 1.5 JVM creates a shared class cache persisted to the disk in a /tmp directory named as "javasharedresources." This directory typically contains all the core Java API classes, as well as the core WAS classes. This cache is reused by other JVMs started on the same machine. This is an enhancement to reduce the virtual memory usage with multiple WAS servers running on the same server. Also, it reduces the startup time for subsequent VMs created. However, since the APM agent inserts probes into the native/core Java classes (for example, threads, sockets), as well the core WAS classes, this could cause the above issues.
For example, if the java.lang.Thread class is substituted with a call to the ManagedThread class from the Wily code and this class was persisted to the cache, any other JVM starting on the same machine trying to use the same Thread Class from the shared cache will fail since this core class now has references to the Wily codebase.
JVMs make use of this shared cache feature through Inter Process Commmunication (IPC) and shared memory segments, which is why solutions A and B above work.
Wily software uses bytecode manipulation to modify WAS and JVM classes to reference Wily classes. WAS 6.1 uses class sharing to cache the contents of class files to provide time and memory performance benefits. However, if it is populated with bytecodes from classes that have been manipulated by Wily software, then all server JVMs will attempt to load Wily classes. If Wily software is not active on a particular server, then failures such as this can occur. Use one of the following to resolve this:
Note: This is possible only with Websphere 6.1 as the class sharing feature is available only this release..
This shared class feature is controlled through a JVM argument -Xshareclasses[:name=<cachename>] passed on the command line to the JVM.
Options to address this issue are as follows: