DA cannot be accessed after upgrade to DX Netops 20.2.8 Performance Management in a Fault Tolerant Data Aggregator environment
When I try and put either DA into active or maintenance state there is an error similar to:
cat: /DASharedRepo/acl-token.properties: No such file or directory:
Where /DASharedRepo is the shared directory for the Data Aggregators.
Also in the /opt/IMDataAggregator/InstallLogs_ConsulServer/CA_Installer_for_Consul_Install_MM_DD_YYYY_HH_MM_SS.log we see errors such as:
Additional Notes: ERROR - Error while boostrapping ACL: Request to Bootstrap ACL failed. ACL might be in legacy mode or might have already been configured HTTP Error Code: 500, response: Internal Server Error
Custom Action: com.ca.im.consul.ext.installer.customercode.BootstrapAcl
Status: FATAL ERROR
Additional Notes: FATAL ERROR - class com.ca.im.consul.ext.installer.customercode.BootstrapAcl FatalInstallException: HTTP Error Code: 500, response: Internal Server Error
When upgrading the Fault Tolerant Data Aggregators either:
1) The Data Aggregator Proxy has not been upgraded.
2) The DA Proxy was upgraded after the Data Aggregators
Per the documentation, the Data Aggregator Proxy MUST be installed/upgraded before the Fault Tolerant Data Aggregators:
This becomes very important due to the Enhanced Consul Security Using ACLs (20.2.8 and Higher Only) noted here:
Release : 20.2.8
Component : IM Reporting / Admin / Configuration
1) If you have not done so already upgrade the Data Aggregator Proxy.
2) Re-run the Data Aggregator installer over the top of one of the existing Fault Tolerant Data Aggregator installs.
(Do NOT uninstall the Data Aggregator before re-running the installer!)
This will set up the consul ACLs as required
Note this also may be seen if a shared data directory is not being used, this is required for a Fault Tolerant Data Aggregator per the documentation.