Post Upgrade NetOps 20.2. Unable to start DA: Error querying COnsul agent: Unexpected response code:500

Products

• CA Infrastructure Management
• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

Upgrading NetOps 20.2.3; steps followed:

Data Aggregators DA1 and DA2 down

ERROR:   Error querying Consul agent: Unexpected response code: 500
InstallLogs_Consul-ext]$ /opt/IMDataAggregator/consul/bin/consul kv get -recurse
Error querying Consul agent: Unexpected response code: 500
[ InstallLogs_Consul-ext]$ /opt/IMDataAggregator/scripts/dadaemon activate
 InstallLogs_Consul-ext]$ /opt/IMDataAggregator/consul/bin/consul kv get -recurse
Error querying Consul agent: Unexpected response code: 500

Error querying Consul agent: Unexpected response code: 500
InstallLogs_Consul-ext]$ /opt/IMDataAggregator/consul/bin/consul kv get -recurse
Error querying Consul agent: Unexpected response code: 500
InstallLogs_Consul-ext]$ /opt/IMDataAggregator/scripts/dadaemon activate
InstallLogs_Consul-ext]$ /opt/IMDataAggregator/consul/bin/consul kv get -recurse
Error querying Consul agent: Unexpected response code: 500

No cluster leader

Fault tolerant data aggregator (DA) does not communicate with consul 

Environment

Release : 20.2

Component : IM Reporting / Admin / Configuration

Cause

Sometimes these directories can get bad data in them and cause communication issues 

Resolution

cleared/removed the data directories and files from the the systems . 

on the consul clear
/opt/CA/daproxy/data

On the DA's clear
/opt/IMDataAggregator/consul/data