PAM-CLNT-0007: Session disconnected due to a problem with session recording and PAM-SPFD-0014: Failed to launch connection as the session recording can not be started.
Article ID: 190003
Updated On:
Products
CA Privileged Access Manager (PAM)
CA Privileged Access Manager - Cloakware Password Authority (PA)
CA Privileged Access Manager - Server Control (PAMSC)
Issue/Introduction
All out of sudden PAM 3.2.x does not allow access to target servers and report following errors in the Sessions Log.
PAM-CLNT-0007: Session disconnected due to a problem with session recording.
PAM-SPFD-0014: Failed to launch connection as the session recording can not be started.
And get a popup error.
PAM-CLNT-0007: Session disconnected due to a problem with session recording.
PAM-SPFD-0014: Failed to launch connection as the session recording can not be started.
And get a popup error.
Environment
Release : 3.2.x, 3.3.0, 3.3.1
Component : PRIVILEGED ACCESS MANAGEMENT
Cause
Session Recording is hosted on JBOSS Application Server on PAM.
Although PAM 3.2.x are hosted on a 64bit Debian Linux, PAM 3.2.x inherited JVM configurations from PAM 2.x which was based on 32bit OS.
As a result, PAM 3.2.x (including 3.3.0 and 3.3.1) are configured with 2GB RAM for JBOSS where the max memory consumption can exceed the limit.
In the JBOSS server.log, following exception is reported.
2020-04-27 13:49:05,127 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.SecureLayer.findChannelProcessor(SecureLayer.java:172)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.SecureLayer.findProcessor(SecureLayer.java:154)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:85)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:105)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:105)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.ITULayer.doProcessing(ITULayer.java:167)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.layer.ProcessorITULayer.doProcessing(ProcessorITULayer.java:44)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.ITULayer.mainLoop(ITULayer.java:155)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.app.Processor.processServerToClient(Processor.java:147)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.app.Processor.inMemoryMainLoop(Processor.java:85)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.app.Processor.main(Processor.java:323)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.xceedium.gk.ws.impl.ProcessRDPRecordingServiceImpl.processRecording(ProcessRDPRecordingServiceImpl.java:90)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.xceedium.gk.ws.impl.CompletePartialRecordingTask.run(CompletePartialRecordingTask.java:78)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.FutureTask.run(FutureTask.java:262)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.lang.Thread.run(Thread.java:745)
2020-04-27 13:49:05,215 ERROR [STDERR] 2020-04-27 13:49:05.215 [ERROR] Processor.70 java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:14:58,344 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:01,284 ERROR [org.mortbay.log] EXCEPTION
java.lang.OutOfMemoryError: Java heap space
at java.util.HashMap.newKeyIterator(HashMap.java:968)
at java.util.HashMap$KeySet.iterator(HashMap.java:1002)
at java.util.HashSet.iterator(HashSet.java:170)
at sun.nio.ch.Util$2.iterator(Util.java:303)
at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:537)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:190)
at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:706)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)
2020-04-27 14:15:02,777 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:01,284 ERROR [org.mortbay.log] EXCEPTION
java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:07,874 ERROR [STDERR] 2020-04-27 14:15:07.094 [ERROR] Processor.104 java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:12,172 ERROR [STDERR] Exception in thread "btpool0-2"
2020-04-27 14:15:12,172 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:25:58,307 ERROR [STDERR] Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0"
2020-04-27 14:25:58,391 ERROR [STDERR] Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2"
Although PAM 3.2.x are hosted on a 64bit Debian Linux, PAM 3.2.x inherited JVM configurations from PAM 2.x which was based on 32bit OS.
As a result, PAM 3.2.x (including 3.3.0 and 3.3.1) are configured with 2GB RAM for JBOSS where the max memory consumption can exceed the limit.
In the JBOSS server.log, following exception is reported.
2020-04-27 13:49:05,127 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.SecureLayer.findChannelProcessor(SecureLayer.java:172)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.SecureLayer.findProcessor(SecureLayer.java:154)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:85)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:105)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:105)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.ITULayer.doProcessing(ITULayer.java:167)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.layer.ProcessorITULayer.doProcessing(ProcessorITULayer.java:44)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.core.layer.ITULayer.mainLoop(ITULayer.java:155)
2020-04-27 13:49:05,127 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.app.Processor.processServerToClient(Processor.java:147)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.app.Processor.inMemoryMainLoop(Processor.java:85)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.ca.xsuite.app.rdp3.processor.app.Processor.main(Processor.java:323)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.xceedium.gk.ws.impl.ProcessRDPRecordingServiceImpl.processRecording(ProcessRDPRecordingServiceImpl.java:90)
2020-04-27 13:49:05,215 ERROR [STDERR] at com.xceedium.gk.ws.impl.CompletePartialRecordingTask.run(CompletePartialRecordingTask.java:78)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.FutureTask.run(FutureTask.java:262)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2020-04-27 13:49:05,215 ERROR [STDERR] at java.lang.Thread.run(Thread.java:745)
2020-04-27 13:49:05,215 ERROR [STDERR] 2020-04-27 13:49:05.215 [ERROR] Processor.70 java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:14:58,344 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:01,284 ERROR [org.mortbay.log] EXCEPTION
java.lang.OutOfMemoryError: Java heap space
at java.util.HashMap.newKeyIterator(HashMap.java:968)
at java.util.HashMap$KeySet.iterator(HashMap.java:1002)
at java.util.HashSet.iterator(HashSet.java:170)
at sun.nio.ch.Util$2.iterator(Util.java:303)
at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:537)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:190)
at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:706)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)
2020-04-27 14:15:02,777 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:01,284 ERROR [org.mortbay.log] EXCEPTION
java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:07,874 ERROR [STDERR] 2020-04-27 14:15:07.094 [ERROR] Processor.104 java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:12,172 ERROR [STDERR] Exception in thread "btpool0-2"
2020-04-27 14:15:12,172 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:25:58,307 ERROR [STDERR] Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0"
2020-04-27 14:25:58,391 ERROR [STDERR] Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2"
Resolution
This is resolved in PAM 3.3.2 as all the JVM settings have been updated.
PAM 3.3.2 now has 8GB RAM configured for JBOSS.
A workaround is to restart JBOSS with help from support but this problem will resurface until it has been upgraded to PAM 3.3.2 or higher.
It is advisable to schedule after hours to perform this to prevent interrupting or losing active sessions recording.
PAM 3.3.2 now has 8GB RAM configured for JBOSS.
A workaround is to restart JBOSS with help from support but this problem will resurface until it has been upgraded to PAM 3.3.2 or higher.
It is advisable to schedule after hours to perform this to prevent interrupting or losing active sessions recording.
Feedback
Yes
No
Powered by
