JCP is not stopping gracefully after the stop signal of the Service Manager

Products

• CA Automic Workload Automation - Automation Engine
• Automic Automation
• Automic SaaS
• CA Automic One Automation

Issue/Introduction

During a server's maintenance shutdown, the Service Manager is stopped by systemd but the JCP doesn't stop gracefully.

Consequently, the JCP is not removed from MQSRV, which may lead to Agents in an inconsistent state: Agents connected to the JCP may not detect their disconnection from the system and remain connected in AWI. As a consequence, no jobs are launched but the Agent doesn't reconnect either.

In the Service Manager log, the stop of the JCP is triggered:

20240807/114605.393 - U00022010 Terminate process 'JCP/UC4 JCP-Server [AWA_PROD#CPxxx] - 222 Connections' (ID '1697') with mode 'C'.

A thread of the JCP receives the stop signal but stops only 90 seconds later (killed by systemd) because other threads are still busy:

...
20240807/114736.435 - 31                at java.lang.Thread.run(Thread.java:750)
<EOF> 

In the meantime, forced traces are generated:

20240807/114606.032 - 31               com.automic.kernel.osgi.NoServiceException: No registered service for 'com.automic.kernel.impl.busy.BusyValuesFacade', filter:null
20240807/114606.032 - 31                   at com.automic.kernel.osgi.OSGIRegistryFunctions.lookup(OSGIRegistryFunctions.java:44)
20240807/114606.032 - 31                   at com.automic.kernel.osgi.Service.lookup(Service.java:85)
20240807/114606.032 - 31                   at com.automic.kernel.osgi.Service.lookup(Service.java:69)
20240807/114606.032 - 31                   at com.automic.kernel.impl.MessageRuntimeMonitor.<init>(MessageRuntimeMonitor.java:40)
20240807/114606.032 - 31                   at com.automic.kernel.impl.StructuredMessageHandler.handleMessageForCpClient(StructuredMessageHandler.java:165)
20240807/114606.032 - 31                   at com.automic.kernel.impl.StructuredMessageHandler.execute(StructuredMessageHandler.java:88)
20240807/114606.032 - 31                   at com.automic.kernel.impl.NetworkMessageDispatcher$NetworkMessageRunnable.runSave(NetworkMessageDispatcher.java:117)
20240807/114606.032 - 31                   at com.automic.kernel.impl.ExecutorTaskWrapper.run(ExecutorTaskWrapper.java:37)
20240807/114606.032 - 31                   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
20240807/114606.032 - 31                   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
20240807/114606.032 - 31                   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
20240807/114606.032 - 31                   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
20240807/114606.032 - 31                   at java.lang.Thread.run(Thread.java:750)

Environment

Release: 21.x 24.x

Component: Automation Engine

Cause

DE153361

Resolution

Update to a fix version listed below or a newer version if available.

Fix version:

Component(s): Automation Engine

Automation.Engine 24.3.0 - Available

Additional Information

Desc: A problem has been fixed in the JCP where forced traces have been produced during shutdown of the system.