ActiveMQ not starting after an abnormal stop of server
Article ID: 191765
Updated On:
Products
CA Service Management - Service Desk Manager
Process Automation Manager
CA Process Automation Base
Issue/Introduction
Sometimes, when an abnormal stop happens on the servers (electricity issues, VM reboot, etc.) and some installed features may be impacted.
If after an abnormal stop PAM's login page is not shown (even though the Orchestrator service starts), the problem is caused by ActiveMQ module not starting correctly.
In c2o.log the following can be seen:
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Initialization] Starting ActiveMQ:
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Initialization] Checking if ActiveMQ has been started...
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] activeMqExtendedMemory 512m ====== 512m
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] [Debug only]Starting ActiveMQ passing PID:6700, JAVA.HOME=e:\apps\java\jdk1.8.0_211\jre
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] ActiveMQ completed with exitStatus: 0
ERROR [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] Unable to start ActiveMQ or it has been stopped...
ERROR [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] PAM Server will shut down now
If after an abnormal stop PAM's login page is not shown (even though the Orchestrator service starts), the problem is caused by ActiveMQ module not starting correctly.
In c2o.log the following can be seen:
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Initialization] Starting ActiveMQ:
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Initialization] Checking if ActiveMQ has been started...
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] activeMqExtendedMemory 512m ====== 512m
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] [Debug only]Starting ActiveMQ passing PID:6700, JAVA.HOME=e:\apps\java\jdk1.8.0_211\jre
INFO [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] ActiveMQ completed with exitStatus: 0
ERROR [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] Unable to start ActiveMQ or it has been stopped...
ERROR [com.optinuity.c2o.c2oserver.ServerManager] [ Thread-13] PAM Server will shut down now
Environment
Process Automation 4.3.x
Cause
With this information, ActiveMQ logs (activemq.log) must be reviewed, and the following message may be seen:
ERROR | Failed to load: file [E:\Apps\PAM\activemq\conf\broker-config.xml], reason: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in file [E:\Apps\PAM\activemq\conf\broker-config.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: Failed to start job scheduler store: JobSchedulerStore: activemq-data\Node0\scheduler | org.apache.activemq.xbean.XBeanBrokerFactory | main
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in file [E:\Apps\PAM\activemq\conf\broker-config.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: Failed to start job scheduler store: JobSchedulerStore: activemq-data\Node0\scheduler
Caused by: java.lang.RuntimeException: Failed to start job scheduler store: JobSchedulerStore: activemq-data\Node0\scheduler
Caused by: java.io.FileNotFoundException: activemq-data\Node0\scheduler\scheduleDB.redo (The file or directory is corrupted and unreadable)
ERROR | Failed to load: file [E:\Apps\PAM\activemq\conf\broker-config.xml], reason: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in file [E:\Apps\PAM\activemq\conf\broker-config.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: Failed to start job scheduler store: JobSchedulerStore: activemq-data\Node0\scheduler | org.apache.activemq.xbean.XBeanBrokerFactory | main
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in file [E:\Apps\PAM\activemq\conf\broker-config.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: Failed to start job scheduler store: JobSchedulerStore: activemq-data\Node0\scheduler
Caused by: java.lang.RuntimeException: Failed to start job scheduler store: JobSchedulerStore: activemq-data\Node0\scheduler
Caused by: java.io.FileNotFoundException: activemq-data\Node0\scheduler\scheduleDB.redo (The file or directory is corrupted and unreadable)
Resolution
The error seen on logs means that after the abnormal stop, ActiveMQ database got corrupted and is unable to be used.
The easy way to solve this is to stop Orchestrator service, verify that no Java.exe is running on the server, then navigate to ..\CA\PAM\activemq and rename "activemq-data" to something else.
Now start Orchestrator service again and ActiveMQ database should get recreated.
The easy way to solve this is to stop Orchestrator service, verify that no Java.exe is running on the server, then navigate to ..\CA\PAM\activemq and rename "activemq-data" to something else.
Now start Orchestrator service again and ActiveMQ database should get recreated.
Additional Information
If this doesn't help, probably would be better to "Reinstall" Process Automation as a way to let the installer re create all ActiveMQ folders.
Feedback
Yes
No
Powered by
