VAMI shows Cluster Health: Read Only Primary or Degraded state

Products

• VMware Cloud Director

Issue/Introduction

• After a failed upgrade snapshot revert or increasing the capacity of the embedded PostgreSQL database on VMware Cloud Director Appliance by following the Increase PostgreSQL database disk , the Cluster Health shows DEGRADED or READ ONLY PRIMARY
  
  
  
  
  NOTE: '!' mark before the node means - standby node is detached from its upstream primary.
  
  
• Cannot access VMware Cloud Director UI, instead getting the below error when trying to access the page:
  
  HTTP ERROR 404 JSP file [/error.jsp] not found
  
  
• Upgrade process failed and now the service will not start
• NFS Server used for the Transfer Server Storage was in-accessible and now the VAMI shows "READ ONLY PRIMARY"

Environment

VMware Cloud Director 10.x

Cause

• Cluster Health shows DEGRADED when the primary and one of the standby cells are online and operational, but the other standby cell is non-functional. The primary database is functional in this state, but if there is another database failure of either of the operational cells, the primary will become non-functional. The non-functional standby cell must be replaced with a new, functioning standby cell as soon as possible to restore the cluster to a Healthy state. The VMware Cloud Director UI and API are functional.
• Cluster Health shows READ ONLY PRIMARY when there is an online primary database, but it is READ_ONLY because the environment does not have an operational standby cell. Two new standby cells must be deployed. The VMware Cloud Director UI and API are not available.

Resolution

• To workaround the issue please use the following KB Article.

Reverting from a Cloud Director Appliance Deployment with Database HA to a Cloud Director Appliance Deployment Without Database HA

• If the above does not work please unregister the faulty Standby cells using the following document: Unregister a Standby Cell in Your VMware Cloud Director Database High Availability Cluster and make VAMI HEALTHY with only Primary cell.
• After removing the faulty standby cells successfully you can redeploy the cells again using the following document: Configure Your VMware Cloud Director Standby and Application Cells
• If the above workaround  do not help, please contact technical support and note this Knowledge Article ID (384987) in the problem description. For more information, see How to Submit a Support Request.