OneClick WebApp Occasionally Hangs and Doesn't Fully Launch - deadlock
search cancel

OneClick WebApp Occasionally Hangs and Doesn't Fully Launch - deadlock


Article ID: 259589


Updated On:


CA Spectrum DX NetOps


We've noticed that our Spectrum users who use WebApp to launch OneClick sometimes can't get it to launch.  The only solution is to kill the 2 webtomcat processes and then
    restart it.  A few minutes later, WebApp is then functional.  This problem happens periodically once every 1 - 2 weeks.


Release : 22.2


This was due to a bug in the 3rd party Webswing application whereby a deadlock condition can occur that prevents the Webtomcat from working and leaving users
   unable to launch the WebApp.

Generating java stack dumps against the webtomcat process shows the deadlock.

Analysis of DeadLock


Fixed in Spectrum 22.2.7 which ships WebSwing 22.2.5 which resolves the periodic deadlock issue. 

Spectrum 22.2.2-4 ships with WebSwing 22.2.4, however, this is fixed in 22.1.10 and newer

The WebSwing deadlock has been fixed in version 22.1.10
See issue #755:


Additional Information


To diagnose the hang one needs to generate Java Stack dumps by running (linux) kill -3 against the webtomcat pid.

Note: We do not want the xvfb-run process, we need the PID of the  java process

     kill -3 1834

this is output to standard output which for webtomcat is


We will also need 4 - 5 stack dumps about 60 seconds apart. This will show if the threads are in the same spot. In this case, we were able to 
    see  threads blocked and find one of the threads has holding a lock that another was waiting for while this thread was waiting for the lock
    the other had caused the deadlock.

The procedure for this is as a root user in a bash shell please enter this command 
pstack <webtomcat Pid> >> pstack.out 

Please run this 5 times in 1 minute increments, once this has been completed please provide the pstack.out file and the <$SPECROOT>/webtomcat/logs/catalina.out to support.