Steps to Troubleshoot Tomcat issues
Article ID: 20604
Updated On:
Products
Issue/Introduction
Description:
The CA Service Desk Manager Integrations Vol. 2 Greenbook contains information on how to activate Traces on Tomcat when more specific troubleshooting is necessary.
This document explains the expected sequence of steps to have the traces and monitoring done, as well as some additional tips on getting the correct documentation. The main topics are:
Solution:
The steps are explained in the Troubleshooting Problems When Tomcat Hangs or Crashes topic found on page 73 of the Green book:
Preparing the environment
- It is necessary to have pslist installed to monitor the javaw.exe process usage (memory and CPU).
If your environment doesn't have it installed, this is the time to do it.
The topic Enable Interval Logging on CA SDM Primary Server on page 76 contains the link from where it can be downloaded. After you have gone through all the steps, you should run pslist manually at the prompt to accept the License Agreement. This needs to be done only the first time that pslist is issued on the machine.
NOTE: For SDM 12.7 CA SDM, the performance monitoring is automatically set, unless it had been disabled. You should confirm whether or not this monitoring is active.
- A specific tool is necessary to get a dump from a javaw.exe hang or crash.
The topic Take a Dump of the Process has the link where the free tool named USERDUMP can be downloaded.
However, if you are running Windows 2008 64 bit, you will need to get ADPlus to generate the dump. KB286350 contains the link where to download it from and instructions on how to use it:
How to use ADPlus.vbs to troubleshoot 'hands" and "crashes"
NOTE: It requires .Net to be installed on the server. This might require a reboot.
Preparing Service Desk Manager to get the traces and start the monitoring
To correctly set and activate the traces, follow the instructions from the Green book, starting at page 74:
- Disable Existing pdm_logstat Logging , page 74.
- Enable Tomcat Debug Logging , page 74.
- Create axis.log and jsrvrbop.log Files , page 74.
- Enable Tracing on the CA SDM Server , page 75.
- Enable Interval Logging on the CA SDM Primary Server , page 76.
Generating a Dump
Follow some considerations about the types of dumps and how to get them.
If Tomcat is crashing, follow the instructions on the respective Microsoft KB on how to obtain a crashing dump. Usually you need to have the tool running as soon as the Tomcat process is running so the dump can be obtained when the process dies.
If Tomcat is hanging, the tool is used by the time the problem is seen.
If Tomcat is not consuming CPU and you are not sure it is really in hang mode, test Web Services prior to getting the dump.
This can be done by following the steps on topic Execute USD_WS_R12_JWS.exe to Test Tomcat Responsiveness from the Greenbook on page 78. As stated in the Greenbook, "if this application responds during the problem, it indicates that Tomcat is not hanging. The problem is with the application and not necessarily with the CA SDM Web Services." and such no dump needs to be generated.
Some examples on how to get the hang dump using the process name:
If running ProcDump: procdump -h javaw.exe javaw_<timestamp>.dmp
If running ADPlus: adplus -hang -pn javaw.exe -o c:\temp\javaw_<timestamp>.dmp
If running UserDump: userdump javaw
In case you have more than one javaw.exe running on your SDM installation (Visualizer and Support Automation have their own processes), you might want to use the PID in place of the process name. To obtain the correct PID proceed as follows:
- From the SDM stdlog files:
nxcd log
type stdlog* | find /i "SERVICEDESK Tomcat"
it should return a line as the following:
09/24 16:11:03.45 mysdmserver pdm_tomcat 6976 SIGNIFICANT pdm_tomcat.c 1130 SERVICEDESK Tomcat javaw process started (pid=5384)
- From Windows:
C:\>WMIC PROCESS get Caption,Commandline,Processid | findstr javaw.exe | findstr CATALINA_BASE\
Which produces:
javaw.exe "C:\Program Files (x86)\CA\SC\jre\1.6.0_23\bin\javaw"-Xrs -javaagent:c:\wily\Agent.jar -Djava.net.preferIPv4Stack=false -Dcom.wily.introscope.agentProfile=c:\wily\core\config\IntroscopeAgent.profile -Dcom.wily.introscope.agent.agentName=SDM_BASE_TOMCAT -Xms64M -Xmx512M -Xdebug -Xrunjdwp:transport=dt_socket,address=1044,server=y,suspend=n -Djaas.config=C:\PROGRA~2\CA\SERVIC~1\add-ons\caflow\jaas.config -Djava.security.auth.login.config=C:\PROGRA~2\CA\SERVIC~1\add-ons\caflow\jaas.config -Djdbc.baseDriver=com.microsoft.sqlserver.
jdbc.SQLServerDriver -Djavax.xml.transform.TransformerFactory=com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl -Dsun.io.useCanonCaches=false
-Djava.awt.headless=true -Djava.endorsed.dirs="C:\Program Files (x86)\CA\SC\tomcat\6.0.30"\common\endorsed -classpath "C:\Program Files (x86)\CA\SC\tomcat\6.0.3
0"\common\lib\tools.jar;"C:\Program Files (x86)\CA\SC\jre\1.6.0_23"\java\lib\tools.jar;C:\PROGRA~2\CA\SERVIC~1\java\lib\tools.jar;"C:\Program Files (x86)\CA\SC\
tomcat\6.0.30"\bin\bootstrap.jar;C:\PROGRA~2\CA\SERVIC~1\java\lib\sqljdbc.jar;C:\PROGRA~2\CA\SERVIC~1\java\lib\ojdbc14.jar -Dcatalina.base=C:\PROGRA~2\CA\SERVIC
~1\bopcfg\www\CATALINA_BASE -Dcatalina.home="C:\Program Files (x86)\CA\SC\tomcat\6.0.30" -Djava.io.tmpdir=C:\PROGRA~2\CA\SERVIC~1\bopcfg\www\CATALINA_BASE\temporg.apache.catalina.startup.Bootstrap start 4200
Where:
CATALINA_BASE indicates this is the Service Desk Tomcat.
4200 is the port number.
Some examples on how to get the hang dump using the process name:
If running ProcDump: procdump -h <pid> javaw_<timestamp>.dmp
If running ADPlus: adplus -hang -p <pid> -o c:\temp\javaw_<timestamp>.dmp
If running UserDump: userdump <pid>
Deactivate the traces
After all documentation had been generated, deactivate the traces.
The steps are on page 77 of the Greenbook, on topic Disable Tracing.
Send the documentation to CA Support
First of all, get information from javaw. Follow the steps under topic Create a Dump of javaw on page 76.
Next, send all documentation requested on topic Send the Dump and Traces to CA Technologies Support, along with any important detail verified during the gathering proceeding.
Environment
Component:
Attachments
Feedback
