SDDC Manager UI is stuck on the "VMware Cloud Foundation is initializing" screen (VMware Cloud Foundation 4.x, 5.x, 9.x)"

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

•   SDDC Manager UI fails to load 
  
  
• VMware Cloud Foundation is initializing

less var/log/vmware/vcf/sddc-manager-ui-app/sddcManagerServer.log

YYYY-MM-DD ERROR [services/wrappers/requestPromiseWrapper.js, init-pscs-primary, logAxiosError:51] axios.error.config {"config":{"adapter":["xhr","http"],"env":{},"headers":{"Accept":"application/json, text/plain, */*","Accept-Encoding":"gzip, compress, deflate, br","User-Agent":"axios/1.6.2","X-B3-SpanId":"46e3ccf847804baa","X-B3-TraceId":"faf873e40ed6472d"},"maxBodyLength":-1,"maxContentLength":-1,"method":"get","timeout":0,"transformRequest":[null],"transformResponse":[null],"transitional":{"clarifyTimeoutError":false,"forcedJSONParsing":true,"silentJSONParsing":true},"url":"http://127.0.0.1/commonsvcs/idpMetadata","xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN"}}
YYYY-MM-DD ERROR  [services/wrappers/requestPromiseWrapper.js, init-pscs-primary, logAxiosError:56] axios.error.response.status {"status":500}

YYYY-MM-DD ERROR  [services/pscUtils.js, init-pscs, attemptPSCInit:67] Caught error from await primaryPscInit
YYYY-MM-DD WARN  [services/pscUtils.js, init-pscs, attemptPSCInitWithRetry:111]
100.109: VError: PSC Initialization attempt "23" failed: Failed to initiate PSC: Primary psc init failed and failover psc init also failed: Unable to retrieve iDP Metadata: Request failed with status code 500
::
Error Info: {"retryCount":23,"status":403,"errorModule":100,"errorCode":109,"code":"ERR_BAD_RESPONSE"}
caused by:
100.108: VError: Failed to initiate PSC: Primary psc init failed and failover psc init also failed: Unable to retrieve iDP Metadata: Request failed with status code 500
::
Error Info: {"errorModule":100,"errorCode":108,"code":"ERR_BAD_RESPONSE"}
caused by:
100.107: VError: Primary psc init failed and failover psc init also failed: Unable to retrieve iDP Metadata: Request failed with status code 500
::
Error Info: {"errorModule":100,"errorCode":107,"code":"ERR_BAD_RESPONSE"}
caused by:
100.149: VError: Unable to retrieve iDP Metadata: Request failed with status code 500
::
Error Info: {"errorModule":100,"errorCode":149,"code":"ERR_BAD_RESPONSE"}
caused by:
AxiosError: Request failed with status code 500
All Errors Info:
 {"code":"ERR_BAD_RESPONSE","retryCount":23,"status":403}

Environment

• VMware Cloud Foundation (VCF) 4.x, 5.x, 9.0.x, 9.1.0
• SDDC Manager

 

Cause

This issue occurs when credential table in the SDDC Manager database has inconsistency 

Resolution

Resolution requires clearing/alerting database tables . Please open a ticket with Broadcom support for assistance.

Additional Information

SDDC Manager is unable to SSH into the node(s) in question to run through the attempted workflow due to a mismatch in what it expects the Host key to be vs what is being presented by the node.

SDDC Manager の UI が「VMware Cloud Foundation is initializing」画面で停止する