SDDC password operations for SSO account fail with "Unable to retrieve entity type VCENTER from inventory" and Remediation Message "Check if inventory APIs are up and responding"

Products

• VMware SDDC Manager

Issue/Introduction

SDDC password operations fail with "Unable to retrieve entity type VCENTER from inventory" and Remediation Message "Check if inventory APIs are up and responding"

operationsmanager.log shows:

2024-08-27T13:23:00.864+0000 DEBUG [vcf_om,e63c478d4163e736,07e4]  [c.v.v.p.helper.InventoryHelper,om-exec-14] Fetching details of Vcenter with resource ID: <Resource_ID> from Inventory.
2024-08-27T13:23:00.869+0000 ERROR [vcf_om,e63c478d4163e736,07e4] [c.v.v.p.u.d.NsxtManagerSsoUpdater.om-exec-14] Failed to update NsxT Manager with SSO credentials. {}
...
caused by: com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiException:

Environment

VCF 4.4.1.x

Cause

The vCenter ID referenced in the operationmanager.log does not exist in the vcenter table but is referenced in the platform_credentials table. 

Resolution

1. Take a snapshot of SDDC Manager
2. SSH into SDDC Manager as vcf and su to root
3. Copy/paste the vCenter resource ID from the operationsmanager.log into a notepad. The resource ID should be contained in the entry right above the error message.
4. Log into the SDDC Manager DB by running this command: psql -h localhost -U postgres -d platform
5. Verify the vCenter ID is not referenced in the vcenter table by running this command using the resource ID from the notepad: select * from vcenter where id='<Resource_ID>';
6. Once it is verified that the vCenter ID does not exist in the vcenter table, check if the vCenter ID is referenced in the credential table by running: select * from credential where entityid='<Resource_ID>';
7. If the vCenter ID shows up in the credential table, delete it by running: delete from credential where entityid=''<Resource_ID>';
8. Restart services by running: /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh
9. Retry the password operation