Error: "Domain or some of its components are not in active state" during Upgrade Planning in VCF 9.1

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• When attempting to plan a component upgrade in the VMware Cloud Foundation (VCF) Operations, the planning phase fails and triggers the following error message in the UI:
  Saving domain target version failed. Validation failures occurred while trying to save the target version: Domain <name> or some of its components are not in active state
  
  

Environment

VMware Cloud Foundation (VCF) 9.1

Cause

This issue occurs when a target Workload Domain component, such as the NSX-T Edge Cluster, is stuck in an ERROR state in the SDDC Manager database. The upgrade pre-checks mandate that all domain components must reflect an ACTIVE status in the database before an upgrade can be planned or initiated.

Resolution

To resolve this issue, manually update the status of the affected component in the SDDC Manager Platform database.

Prerequisites:

Important: Before making any direct modifications to the SDDC Manager database, take a snapshot of the SDDC Manager appliance.

1. Validate the status in the SDDC Manager UI:
   
   1. Login to the NSX Manager.
   2. Review the status of the NSX-T Edge Cluster and its associated Edge Nodes.
      Note: Ensure the Edge Nodes are healthy and operational in both SDDC Manager and NSX-T Manager. If there is a legitimate underlying issue with the edges, resolve that first. If they are healthy but stuck in an ERROR state in SDDC Manager, proceed to the next steps to clear the stale database entry.
      
      
2. SSH into the SDDC Manager appliance using the vcf user, and then su to root.
   
   
3. Connect to the platform database by running the following command:
   /usr/pgsql/13/bin/psql -U postgres -h localhost -d platform


4. Identify the affected NSX-T Edge Cluster:
   1. Run the following SQL query to locate edge clusters that are not in an active state.
      SELECT id, status FROM nsxt_edge_cluster WHERE status != 'ACTIVE';
   2. Take note of the id string returned for the component in the non-active/ERROR state.
      
      
5. Update the component status:
   1. Execute the following UPDATE statement, replacing <ERROR ID state> with the actual ID retrieved in the previous step:
      UPDATE nsxt_edge_cluster SET status = 'ACTIVE' WHERE id = '<ERROR state ID>';


6. Exit the database: \q
   
   
7. Restart SDDC Manager services: Run the built-in script to restart the services and apply the database changes:
   /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh


8. Once the services have fully restarted, log back into the SDDC Manager UI.
   
   
9. Navigate to the Updates/Patches tab and re-run the component upgrade planning. The planning phase should now complete successfully without the validation failure.