ActiveMQ on Data Aggregator of DX NetOps Performance Management failing to start after upgrade

Products

• CA Performance Management
• Network Observability

Issue/Introduction

After upgrading DX NetOps Performance Management, the Data Aggregator (or Data Collector) upgrade installation shows: Successful with errors.

Unable to start ActiveMQ service on Data aggregator, for example;

root@host Logs]# service activemq status
Redirecting to /bin/systemctl status activemq.service
● activemq.service - Apache ActiveMQ
   Loaded: loaded (/etc/systemd/system/activemq.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2021-06-22 19:39:19 EDT; 42s ago
  Process: 12311 ExecStop=/opt/IMDataAggregator/scripts/activemq stop sysd (code=exited, status=0/SUCCESS)
  Process: 29148 ExecStart=/opt/IMDataAggregator/scripts/activemq start sysd (code=exited, status=0/SUCCESS)
 Main PID: 29235 (code=exited, status=1/FAILURE)

Jun 22 19:39:19 host systemd[1]: Starting Apache ActiveMQ...
Jun 22 19:39:19 host activemq[29148]: Starting ActiveMQ
Jun 22 19:39:19 host activemq[29148]: INFO: Loading '/opt/IMDataAggregator/broker/apache-activemq//bin/env'
Jun 22 19:39:19 host activemq[29148]: INFO: Using java '/opt/IMDataAggregator/jre/bin/java'
Jun 22 19:39:19 host activemq[29148]: INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
Jun 22 19:39:19 host activemq[29148]: INFO: pidfile created : '/opt/CA/IMDataAggregator/broker/apache-activemq//data/activemq.pid' (pid '29235')
Jun 22 19:39:19 host systemd[1]: Started Apache ActiveMQ.
Jun 22 19:39:19 host systemd[1]: activemq.service: main process exited, code=exited, status=1/FAILURE
Jun 22 19:39:19 host systemd[1]: Unit activemq.service entered failed state.
Jun 22 19:39:19 host systemd[1]: activemq.service failed.

 

root@host Logs]# service activemq status
Redirecting to /bin/systemctl status activemq.service
● activemq.service - Apache ActiveMQ
   Loaded: loaded (/etc/systemd/system/activemq.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2021-06-22 19:39:19 EDT; 42s ago
  Process: 12311 ExecStop=/opt/IMDataCollector/scripts/activemq stop sysd (code=exited, status=0/SUCCESS)
  Process: 29148 ExecStart=/opt/IMDataCollector/scripts/activemq start sysd (code=exited, status=0/SUCCESS)
 Main PID: 29235 (code=exited, status=1/FAILURE)

Jun 22 19:39:19 host systemd[1]: Starting Apache ActiveMQ...
Jun 22 19:39:19 host activemq[29148]: Starting ActiveMQ
Jun 22 19:39:19 host activemq[29148]: INFO: Loading '/opt/IMDataCollector/broker/apache-activemq//bin/env'
Jun 22 19:39:19 host activemq[29148]: INFO: Using java '/opt/IMDataCollector/jre/bin/java'
Jun 22 19:39:19 host activemq[29148]: INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
Jun 22 19:39:19 host activemq[29148]: INFO: pidfile created : '/opt/IMDataCollector/broker/apache-activemq//data/activemq.pid' (pid '29235')
Jun 22 19:39:19 host systemd[1]: Started Apache ActiveMQ.
Jun 22 19:39:19 host systemd[1]: activemq.service: main process exited, code=exited, status=1/FAILURE
Jun 22 19:39:19 host systemd[1]: Unit activemq.service entered failed state.
Jun 22 19:39:19 host systemd[1]: activemq.service failed.

Cause

Trying to run activemq gives the following in the activemq.out file;

Error: Password file read access must be restricted: <install dir>/broker/apache-activemq/conf/jmx.password

sun.management.AgentConfigurationError

        at sun.management.jmxremote.ConnectorBootstrap.checkPasswordFile(ConnectorBootstrap.java:577)

        at sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:426)

        at sun.management.Agent.startAgent(Agent.java:262)

        at sun.management.Agent.startAgent(Agent.java:452)

Resolution

Change the permissions on the jmx.password file so that only the root user has any form of access;

chmod 600 <install dir>/broker/apache-activemq/conf/jmx.password

This will allow the activemq to start normally and sync to the DX NetOps Portal.