ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

OneClick WebApp high CPU

book

Article ID: 228113

calendar_today

Updated On:

Products

CA Spectrum

Issue/Introduction

today we saw very high CPU load on a OneClick server primarily used for REST subscriptions and only one OneClick WebApp session.  The load increased quite suddenly from less than 5% to over 90% CPU usage.  It was not possible to close the session regularly (closing browser window / issue "Shutdown" command from the WebApp administration console). I had to kill -9 the process, after which the CPU load quickly normalized.

Cause

The REST integration has a high load on this OC server.  Java clients also show performance problems.

 

 

Environment

Release : 20.2

Component : Spectrum OneClick

Resolution

this problem is only seen on the OC dedicated to REST queries and it was decided not to use it for client access.

Additional Information

During the problem out of memory errors are seen n the logs so perhaps increased the RAM dedicated to the webapp may help BUT OC JAVA was also problematic on this server, and the webapp relies on the standard tomcat so it maybe the  same root cause here and cannot be resolved via the webapp.

Right now the webapp is disabled as this server is not used for client access.

If the problem occurs on another server 

1. 
Jstack is for Jthreads dumps.
Use jstack  -F  to force a thread dump the issue is recreated.

cd <specroot>\Java\bin jstack -F <OCserver PID > > stack1.txt 
with one minute back take 3 to 4 stack outputs
 
 e.g.
cd <specroot>\Java\bin
jstack -F <OneClickservice PID > > stack1.txt
wait for a minute
jstack -F < OneClickservice PID> stack2.txt
wait for a minute
jstack -F < OneClickservice PID> stack3.txt
 
  
2. 
Jmap is for java heap memory. 
cd <specroot>\Java\bin
jmap -F <OneClickservice PID > > map1.txt
wait for a minute
jmap -F <OneClickservice PID > > map2.txt
wait for a minute
jmap -F <OneClickservice PID > > map3.txt

3. gather the tomcat log, webtomcat log and webswing log and upload for analysis.