SDDC Manager UI loads with error: Server failed to start. Check the SDDC Manager UI Server logs for more details

Products

• VMware SDDC Manager

Issue/Introduction

• SDDC Manager UI fails to load with the below error:
  
  
  
  

• From SDDC Manager's /var/log/vmware/vcf/sddc-manager-ui-app/sddcManagerServer.log, below log snippets are found:

  YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/sddc-manager/inventory/domains/mgmt-vi-domains(monitoring-panel:monitoring/tasks)"}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/error"}
YYYY-MM-DDTHH:MM:SS ERROR [##############] [services/errorHandling.js, error-server-init, errorStateHandler:47] Server is in error state; serving up error html {"serverErrorState":{"code":null,"state":"error"}}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/"}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/sddc-manager/inventory/domains/mgmt-vi-domains(monitoring-panel:monitoring/tasks)"}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/error"}
YYYY-MM-DDTHH:MM:SS ERROR [##############] [services/errorHandling.js, error-server-init, errorStateHandler:47] Server is in error state; serving up error html {"serverErrorState":{"code":null,"state":"error"}}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/"}
YYYY-MM-DDTHH:MM:SS ERROR [##############] [services/ws1bAuthentication/ws1bAuthenticationService.js, init, getWs1bClientId:95] error during getWs1bClientId Request failed with status code 500 {"body":{"arguments":["Identity Internal Server Error"],"causes":[{"message":"Failed to get the IDP type due to Identity Internal Server Error.","type":"com.vmware.evo.sddc.common.core.error.FederatedBrokerIdentityException"},{"message":"Identity Internal Server Error","type":"com.vmware.evo.sddc.identity.model.error.PscException"},{"message":"Unable to obtain Security Token Service from SSO '<vCenter fqdn>'","type":"com.vmware.evo.sddc.common.services.psc.exception.PscException"},{"message":"Server returned 'request expired' less than 0 seconds after request was issued, but it shouldn't have expired for at least 400


• From SDDC Manager's /var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log, below log snippets are found:

  YYYY-MM-DDTHH:MM:SS INFO  [common,##############,####] [c.v.e.s.i.s.PscInventoryServiceEmbeddedImpl,http-nio-127.0.0.1-####-exec-5] Returned PSCs: [Psc(domainId=########-####-####-####-############, vmName=<vCenter hostname>, vmManagementIpAddress=<vCenter IP Address>, vmHostname=<vCenter fqdn>, isReplica=false, port=443, ssoDomain=<SSO Domain>, subDomain=<SSO Domain>, sshHostKeyType=null, sshHostKey=null, bundleRepoDatastore=lcm-bundle-repo, datastoreName=<Datastore Name>, status=ACTIVE, version=<vCenter server version>, deploymentType=EMBEDDED)]
YYYY-MM-DDTHH:MM:SS INFO  [common,##############,####] [c.v.e.s.i.s.services.PscServiceImpl,http-nio-127.0.0.1-####-exec-5] Creating admin client using psc host <vCenter fqdn>
YYYY-MM-DDTHH:MM:SS WARN  [common,##############,####] [c.v.v.s.a.c.v.impl.AbstractClient,http-nio-127.0.0.1-####-exec-5] ******* WARNING ****** WARNING ****** WARNING *******
YYYY-MM-DDTHH:MM:SS WARN  [common,##############,####] [c.v.v.s.a.c.v.impl.AbstractClient,http-nio-127.0.0.1-####-exec-5] Possible remote API mismatch detected. Operation will continue, but errors are likely.
YYYY-MM-DDTHH:MM:SS WARN  [common,##############,####] [c.v.v.s.a.c.v.impl.AbstractClient,http-nio-127.0.0.1-####-exec-5] ******* WARNING ****** WARNING ****** WARNING *******
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.i.s.IdentityProviderServiceImpl,http-nio-127.0.0.1-####-exec-5] Failed to get the IDP type Identity Internal Server Error
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.i.r.a.c.v.IdentityProviderController,http-nio-127.0.0.1-####-exec-5] Unable to get sddc manager oidc information due to Identity Internal Server Error
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-####-exec-5] [######] SDDCMANAGER_GET_OIDC_INFO_FAILED
com.vmware.evo.sddc.identity.rest.api.error.IdentityRestServiceException: at com.vmware.evo.sddc.identity.rest.api.controller.v1.IdentityProviderController.getSddcWs1bOidcInfo(IdentityProviderController.java:444)
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.i.s.VcfServiceInventoryServiceImpl,http-nio-127.0.0.1-7100-exec-6] Error while trying to retrieve service http://127.0.0.1/sddc-manager-ui-app/about status, 500 Internal Server Error: "<html><EOL>  <head><EOL>    <title>Server failed to start !</title><EOL> <style type=

Cause

The connection between the Management vCenter server and the SDDC Manager is lost due to unexpected instances.

Resolution

• Check the connectivity between the Management vCenter server and the SDDC Manager.
• Reboot the Management vCenter server.
• And then restart SDDC Manager services using below command:
  
  /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh

Additional Information

In some instances, the vCenter server might have lost the IP address details or might have lost the NIC connection.