We've noticed that our Spectrum users who use WebApp to launch OneClick sometimes can't get it to launch. The only solution is to kill the 2 webtomcat processes and then
restart it. A few minutes later, WebApp is then functional. This problem happens periodically once every 1 - 2 weeks.
Release : 22.2
This was due to a bug in the 3rd party Webswing application whereby a deadlock condition can occur that prevents the Webtomcat from working and leaving users
unable to launch the WebApp.
Generating java stack dumps against the webtomcat process shows the deadlock.
Analysis of DeadLock
Fixed in Spectrum 22.2.7 which ships WebSwing 22.2.5 which resolves the periodic deadlock issue.
Spectrum 22.2.2-4 ships with WebSwing 22.1.4, however, this is fixed in 22.1.10 and newer
The WebSwing deadlock has been fixed in version 22.1.10
See issue #755: https://www.webswing.org/docs/22.1/start/news.html#22110-lts
To diagnose the hang one needs to generate Java Stack dumps by running (linux) kill -3 against the webtomcat pid.
Note: We do not want the xvfb-run process, we need the PID of the java process
example:
kill -3 1834
We will also need 4 - 5 stack dumps about 60 seconds apart. This will show if the threads are in the same spot. In this case, we were able to
see threads blocked and find one of the threads has holding a lock that another was waiting for while this thread was waiting for the lock
the other had caused the deadlock.