search cancel

PAM-CLNT-0007: Session disconnected due to a problem with session recording and PAM-SPFD-0014: Failed to launch connection as the session recording can not be started.

book

Article ID: 190003

calendar_today

Updated On:

Products

CA Privileged Access Manager (PAM) CA Privileged Access Manager - Cloakware Password Authority (PA) CA Privileged Access Manager - Server Control (PAMSC)

Issue/Introduction

All out of sudden PAM 3.2.x does not allow access to target servers and report following errors in the Sessions Log.

PAM-CLNT-0007: Session disconnected due to a problem with session recording.
PAM-SPFD-0014: Failed to launch connection as the session recording can not be started.

And get a popup error.

Environment

Release : 3.2.x, 3.3.0, 3.3.1

Component : PRIVILEGED ACCESS MANAGEMENT

Cause

Session Recording is hosted on JBOSS Application Server on PAM.
Although PAM 3.2.x are hosted on a 64bit Debian Linux, PAM 3.2.x inherited JVM configurations from PAM 2.x which was based on 32bit OS.

As a result, PAM 3.2.x (including 3.3.0 and 3.3.1) are configured with 2GB RAM for JBOSS where the max memory consumption can exceed the limit.
In the JBOSS server.log, following exception is reported.

2020-04-27 13:49:05,127 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.SecureLayer.findChannelProcessor(SecureLayer.java:172)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.SecureLayer.findProcessor(SecureLayer.java:154)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:85)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:105)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.Layer.process(Layer.java:105)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.ITULayer.doProcessing(ITULayer.java:167)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.processor.layer.ProcessorITULayer.doProcessing(ProcessorITULayer.java:44)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.core.layer.ITULayer.mainLoop(ITULayer.java:155)
2020-04-27 13:49:05,127 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.processor.app.Processor.processServerToClient(Processor.java:147)
2020-04-27 13:49:05,215 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.processor.app.Processor.inMemoryMainLoop(Processor.java:85)
2020-04-27 13:49:05,215 ERROR [STDERR]  at com.ca.xsuite.app.rdp3.processor.app.Processor.main(Processor.java:323)
2020-04-27 13:49:05,215 ERROR [STDERR]  at com.xceedium.gk.ws.impl.ProcessRDPRecordingServiceImpl.processRecording(ProcessRDPRecordingServiceImpl.java:90)
2020-04-27 13:49:05,215 ERROR [STDERR]  at com.xceedium.gk.ws.impl.CompletePartialRecordingTask.run(CompletePartialRecordingTask.java:78)
2020-04-27 13:49:05,215 ERROR [STDERR]  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
2020-04-27 13:49:05,215 ERROR [STDERR]  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
2020-04-27 13:49:05,215 ERROR [STDERR]  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2020-04-27 13:49:05,215 ERROR [STDERR]  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2020-04-27 13:49:05,215 ERROR [STDERR]  at java.lang.Thread.run(Thread.java:745)
2020-04-27 13:49:05,215 ERROR [STDERR] 2020-04-27 13:49:05.215 [ERROR] Processor.70  java.lang.OutOfMemoryError: Java heap space 
2020-04-27 14:14:58,344 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:01,284 ERROR [org.mortbay.log] EXCEPTION 
java.lang.OutOfMemoryError: Java heap space
 at java.util.HashMap.newKeyIterator(HashMap.java:968)
 at java.util.HashMap$KeySet.iterator(HashMap.java:1002)
 at java.util.HashSet.iterator(HashSet.java:170)
 at sun.nio.ch.Util$2.iterator(Util.java:303)
 at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:537)
 at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:190)
 at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
 at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:706)
 at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)
2020-04-27 14:15:02,777 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:01,284 ERROR [org.mortbay.log] EXCEPTION 
java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:15:07,874 ERROR [STDERR] 2020-04-27 14:15:07.094 [ERROR] Processor.104  java.lang.OutOfMemoryError: Java heap space 
2020-04-27 14:15:12,172 ERROR [STDERR] Exception in thread "btpool0-2" 
2020-04-27 14:15:12,172 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
2020-04-27 14:25:58,307 ERROR [STDERR] Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0" 
2020-04-27 14:25:58,391 ERROR [STDERR] Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2" 

Resolution

This is resolved in PAM 3.3.2 as all the JVM settings have been updated.
PAM 3.3.2 now has 8GB RAM configured for JBOSS.

A workaround is to restart JBOSS with help from support but this problem will resurface until it has been upgraded to PAM 3.3.2 or higher.
It is advisable to schedule after hours to perform this to prevent interrupting or losing active sessions recording.

Attachments