Extremely slow Production performance due to app server deadlock

Products

• Clarity PPM On Premise
• Clarity PPM SaaS

Issue/Introduction

Performance Issue. Oracle Database. Extremely slow Production performance due to app server deadlock.

Steps to Reproduce: 

1. Perform a Studio update to an object 
2. App Server will pile up on blocked threads 

Expected Results: Performance issue, expecting better performance. 

Actual Results: Performance comes to a crawl until app server with blocked threads is restarted. 

This may cause a Studio object lock

Environment

Release : 16.0.1, 16.0.2

Component : Clarity Architecture, Oracle Database, Postgres

Cause

• The Studio action causes the object caches to get flushed, which gets into an event that clears the Hierarchy node cache. 
• This method tries to find related objects to the one being flushed so it tries to get them and will load from persistence. 
• As part of this process a method is called which will get a list of objects and which will do a global check rights call to see which objects the user has rights to see. 
• This will cause the cache controller to put a lock on the user session cache object while it determines if the user has rights. 
• Meanwhile, on another thread a session has timed out (or a user has logged out) and the cache controller has been called to remove that session from the user session cache. 
• The remove method will synchronize on the user session cache map as well and this is causing the lockup. 

 

Resolution

DE65480 fixed in 16.0.3

Defect first introduced in 16.0.1