NetOps Portal Dashboards hang until console service is restarted

Products

• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

Console graphs hung, re-started service and came back.. notice recurring user message in PC log

Our CAPM console hung, we could log in but no data was populating in charts/graphs.. I re-started the caperfcenter_console service and restored functionality.. However checking PC Service log i notice some repeating error message, and not sure if it's related or if it's some issue we need to clear. 

      | Establishing new usersession object for user:  316 invalidating other session.
INFO  | qtp1788615576-66         | 2022-11-29 14:56:07,603 | com.ca.im.portal.services.user.UserSessionServiceImpl          
      | Establishing new usersession object for user:  316 invalidating other session.
INFO  | qtp1788615576-66         | 2022-11-29 14:56:11,990 | com.ca.im.portal.services.user.UserSessionServiceImpl

Message in coming in the log every 3 seconds, first instance I can find is on 11/14.  I will attach a text file of first time I could find it, then also attach current PCService log.. 

DX NetOps Performance Management Portal web server UI Dashboards and Reports are hanging. They do not complete. The caperfcenter_console service required restart in order to resolve the problem.

Environment

All supported DX NetOps Performance Management releases

Cause

Excessive non-UI based user requests caused problems with the console service providing users with the web UI.

Resolution

The following message indicates two different attempts to log in with the same user were initiated at the same time. It will be seen as a result of that action when the tool is configured to disallow multiple logins with the same user. This setting is found in the Web Browser Session Management section of the Administration->Configuration Settings->Security Settings page of the Portal web UI. These features are documented in the Configure Web Browser Session Settings documentation topic.

Without analysis of things like MySql processlist entries or other resource usage analysis during the active issue it's difficult to provide a strong root cause. The logging does indicate a sudden and likely unexpected and unwanted increase in user access generating those messages. These all coming in via the console service, while active users are also using the web UI during work hours, could cause symptoms such as those reported here.

A restart of the caperfcenter_console alone to resolve this suggests this theory is likely the root cause.