After upgrading the SDDC Manager to 5.2, unable to rotate any passwords, remove unwanted or unused licenses, or upgrade the NSX Manager (the next step in the fully upgrade to 5.2), and various other API checks.

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• After upgrading VMware Cloud Foundation (VCF) SDDC Manager to version 5.2, various UI operations and API checks are blocked. Symptoms include the inability to rotate passwords, remove unused licenses, or proceed with the NSX Manager upgrade.
• The SDDC Manager UI displays the following error messages:

   Failed to get allowed functionalities for domains. Failed to retrieve the resource functionalities
  Scheduling immediate update of bundle failed. Unable to determine if functionality is allowed/blocked.
  Failed to delete license key. Unable to determine if functionality is allowed/blocked.
  This functionality is not allowed at the moment. Please check back later.
  
   

The error message in the SDDC logs:

• Error in /var/log/vmware/vcf/lcm/lcm-debug.log

  ERROR [vcf_lcm,##] [c.v.v.f.t.i.FunctionalityToggleApiFilter,http-nio-127.0.0.1-7400-exec-#] Functionality blocked:
  com.vmware.vcf.functionality.toggle.error.FunctionalityToggleException: Unable to determine if functionality is allowed/blocked.
  Caused by: com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiException

• Error in  /var/log/vmware/vcf/operationsmanager/operationsmanager.log

  ERROR [vcf_om,####,6#57] [c.v.v.f.t.i.FunctionalityToggleApiFilter,http-nio-127.0.0.1-7300-exec-#] Functionality blocked:
  com.vmware.vcf.functionality.toggle.error.FunctionalityToggleException: Unable to determine if functionality is allowed/blocked.
  Caused by: com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiException

• Error in  /var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log

  ERROR [common,###,09#1] [c.v.e.s.i.s.VdsInventoryServiceImpl,http-nio-127.0.0.1-7100-exec-#] Inventory Error, Resource ID: {vds id}
  ERROR [common,###,09#1] [c.v.v.r.r.a.c.v.ResourceFunctionalitiesController,http-nio-127.0.0.1-7100-exec-#] Exception occurred when retrieving the resource 
  com.vmware.evo.sddc.inventory.model.InventoryNotFoundException: Resource ID: {vds id} not found in Vds
      at com.vmware.evo.sddc.inventory.dal.sql.client.TypedClientImpl.getEntity(TypedClientImpl.java:344)
      at com.vmware.evo.sddc.inventory.dal.sql.client.VdsClientImpl.getVds(VdsClientImpl.ja

Environment

VMware Cloud Foundation 5.2

Cause

An orphaned or invalid vSphere Distributed Switch (VDS) ID exists in the cluster_and_vds table of the SDDC Manager PostgreSQL database. This inconsistency typically occurs following a vCenter Server restore or rebuild operation.

Resolution

Contact Broadcom Support to assist with a resolution. To work with a customer representative or a Support Engineer, see Contact Support.