SDDC Manager Plan upgrade UI reports error, "Domain ####-####-####-####-############ inventor state is not active", "New update cannot be started while domain or some of its components are not in an active state"

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

SDDC Manager Plan upgrade UI reports error

Domain ####-####-####-####-############ inventor state is not active.

or the error

New update cannot be started while domain or some of its components are not in an active state

Environment

VMware Cloud Foundation 5.x
VMware Cloud Foundation 9.x

Cause

A component in the SDDC inventory is not in an "ACTIVE" state. 

Resolution

1. Identify the component(s) in SDDC Manager which is in ERROR state
   1. SSH to SDDC manager with vcf and su to root
   2. Run VCF VDT to identify the component in ERROR state. - Refer Using the VCF Diagnostic Tool for SDDC Manager 
      Sample output

      INVENTORY STATUS
      
      
              [PASS]    Host Status Check
                          All hosts are in an ACTIVE state.
      
              [PASS]    Domain Status Check
                          All domains are in an ACTIVE state.
      
              [FAIL]    vCenter Status Check
                          ########-####-####-####-########4ce2 | vcsa01.example.com | ERROR
      
      
              [PASS]    PSCs Status Check
                          All PSCs are in an ACTIVE state.
      
              [PASS]    Cluster Status Check
                          All Clusters are in an ACTIVE state.
      
              [PASS]    NSX Manager Status Check
                          All NSX Managers are in an ACTIVE state.

2. Update the component status to ACTIVE in SDDC database
   1. Take Snapshot of SDDC Manager VM
   2. SSH to SDDC manager with vcf and su to root.
   3. Connect to platform database

      psql -h localhost -U postgres -d platform
      
      For SDDC 5.1
      /usr/pgsql/13/bin/psql -h localhost -U postgres -d platform

   4. Update the component status to ACTIVE identified in Step 1(b)
      update <database_table_name> set status='ACTIVE' where id='<id from step 1(b)>';
      Sample

      update vcenter set status='ACTIVE' where id='########-####-####-####-########4ce2';

      Sample If the component is NSX:
      update nsxt set status='ACTIVE' where id='########-####-####-####-########e92d'; 

3. Retry the plan upgrade from the SDDC Manager UI.

Note: Delete SDDC Manager snapshot after confirming that the error no longer appears.

Additional Information

Commands to check the specific tables:

psql -h localhost -U postgres -d platform -c "select id, lb_hostname, status from vrli;"
psql -h localhost -U postgres -d platform -c "select id, lb_hostname, status from vrops;"
psql -h localhost -U postgres -d platform -c "select id, vrslcm_node, status from vrslcm;"
psql -h localhost -U postgres -d platform -c "select id,vm_hostname,status from psc;"
psql -h localhost -U postgres -d platform -c "select id,cluster_fqdn,status from nsxt;"