Data Aggregator fails to start in Fault Tolerant environment

Products

• Network Observability
• CA Performance Management

Issue/Introduction

In a DX NetOps Performance Management Fault Tolerant (FT) environment, one Data Aggregator (DA) node may fail to start or transition to a 'Ready' state even after maintenance scripts are run.

Typical symptoms include:

• DA 1 is active and running.
• DA 2 remains down or fails to respond to ./dadaemon activate.
• Unable to Contact the DAProxy host in Portal Data Sources.
• Consul services may appear running, but communication between the DA and the Proxy is interrupted.

Environment

• DX NetOps Performance Management
• Fault Tolerant Data Aggregator configuration

Cause

The daproxy service on the dedicated Proxy host is stopped or unresponsive. In an FT setup, the Data Aggregator nodes rely on the DAProxy to manage status and coordination; if this service is down, the secondary DA cannot be activated.

Resolution

To resolve this issue, restart the proxy services and then activate the Data Aggregator node.

1. Log into the Proxy Host as root or the install user.
2. Check the status of the daproxy service:
   bash

   systemctl status daproxy

3. If the service is down, restart it:
   bash

   systemctl stop daproxysystemctl start daproxy

4. Log into the Data Aggregator Host that is currently down.
5. Navigate to the scripts directory and activate the daemon:
   bash

   cd /opt/IMDataAggregator/scripts/./dadaemon activate

6. Verify the status:
   bash

   ./dadaemon status