Running DX NetOps version 22.2.9
All users are getting logged out of the NetOps Portal at the same time roughly once per day. We upgraded to 22.2.9 in late June but didn't experience this problem until last week. We never had this happen on earlier versions.
When it happens all users get logged out at the same time. All 4 caperfcenter services are still running and the Portal server has not rebooted.
SSO Configuration/DX NetOps/Single Sign-On:
Anonymous User Enabled: Disabled
Anonymous User ID: 2
Localhost User Sign-In Page Enabled: Disabled
Localhost User Enabled: Disabled
Localhost User ID: 1
Cookie Timeout Minutes: 600
Encryption Decryption Key: <key>
Encryption Algorithm: AES
Failed Sleep Seconds: 3
Remember Me Enabled: Disabled
Remember Me Timeout Days: 1
Virtual Directory: sso
Scheme for Data Sources: https
Port for Data Sources: 8382
The following is seen for the console service when users are logged out but the command line shows the service still running.
STATUS | wrapper | 2023/08/31 08:22:20 | JVM appears hung: Timed out waiting for signal from JVM. Restarting JVM.
STATUS | wrapper | 2023/08/31 08:22:21 | JVM received a signal SIGKILL (9).
STATUS | wrapper | 2023/08/31 08:22:21 | JVM process is gone.
STATUS | wrapper | 2023/08/31 08:22:21 | JVM exited after being requested to terminate.
STATUS | wrapper | 2023/08/31 08:22:26 | JVM process is gone.
We also see this in the console service log leading up to these events.
STATUS | wrapper | 2023/08/30 11:01:39 | Pinging the JVM took 19 seconds to respond.
All supported DX NetOps Performance Management releases
Console service is being starved for memory, running out and restarting, due to insufficient memory allocation during times of high load.
This servers memory availability allowed for a 10 GB increase to the PC console services memory allocation which resolved the issue.
The Modify Maximum Memory Usage for NetOps Portal documentation topic covers the steps for changing memory allocation on Portal web server services.