VMware Aria Operations High Availability (HA) stuck in "Failed to Deactivate" state or stuck in "Activating" state

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• The purpose of the KB is to help troubleshoot the HA to fix cluster stability.

Symptoms:

• The High Availability in Aria Operations is stuck in the "Failed to Deactivate", "Activating" or "Failed to disable" state in the Admin UI which restricts any configuration change in the cluster like Certificate Renewal.
• The Activate/Deactivate HA button is greyed out.
• The cluster can still be in an Online state but it will not perform precisely.

Environment

VMware Aria Operations 8.x

Cause

• It can happen if the services fail to either Activate or Deactivate the HA.

Resolution

1. Take snapshots of all nodes. For steps refer How to take a Snapshot of VMware Aria Operations 
2. SSH to all the nodes using the root credentials.
3. Stop casa on all the nodes by running the command: service vmware-casa stop
4. Open casa.db.script in text edit by using the command: vi /data/db/casa/webapp/hsqldb/casa.db.script
5. Identify the ha_transition_state in the last line of the file
   
   
   
   
6. Change the value from "FAILED_TO_DEACTIVATE' or 'ENABLING' to "NONE" on all the nodes. (Please note the above screenshot has been taken from Aria Operations version 8.12. The value will be "FAILED_TO_DISABLE" in earlier versions.)
7. Save and close the file.
8. Start casa on all the nodes by running the command: service vmware-casa start
9. Bring the cluster online from the admin UI.
10. Delete the snapshots on all nodes once verified that everything is working as expected.

 

Additional Information

How to enable/disable HA when the option is not available in Aria Operations Manager UI

Impact/Risks:

• It will impact the stability of the cluster which will in turn restrict configuration changes like Certificate renewal.