SDDC Manager UI loads with error: Server failed to start. Check the SDDC Manager UI Server logs for more details

Products

• VMware SDDC Manager

Issue/Introduction

• SDDC Manager UI fails to load with the below error:
  
  
  
  

• From SDDC Manager's /var/log/vmware/vcf/sddc-manager-ui-app/sddcManagerServer.log, below log snippets are found:

  YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/sddc-manager/inventory/domains/mgmt-vi-domains(monitoring-panel:monitoring/tasks)"}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/error"}
YYYY-MM-DDTHH:MM:SS ERROR [##############] [services/errorHandling.js, error-server-init, errorStateHandler:47] Server is in error state; serving up error html {"serverErrorState":{"code":null,"state":"error"}}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/"}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/sddc-manager/inventory/domains/mgmt-vi-domains(monitoring-panel:monitoring/tasks)"}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/error"}
YYYY-MM-DDTHH:MM:SS ERROR [##############] [services/errorHandling.js, error-server-init, errorStateHandler:47] Server is in error state; serving up error html {"serverErrorState":{"code":null,"state":"error"}}
YYYY-MM-DDTHH:MM:SS VERBOSE [##############] [servers/errorStateServer/index.js, error-server-init, null:44] Incoming request for error server {"url":"/ui/"}
YYYY-MM-DDTHH:MM:SS ERROR [##############] [services/ws1bAuthentication/ws1bAuthenticationService.js, init, getWs1bClientId:95] error during getWs1bClientId Request failed with status code 500 {"body":{"arguments":["Identity Internal Server Error"],"causes":[{"message":"Failed to get the IDP type due to Identity Internal Server Error.","type":"com.vmware.evo.sddc.common.core.error.FederatedBrokerIdentityException"},{"message":"Identity Internal Server Error","type":"com.vmware.evo.sddc.identity.model.error.PscException"},{"message":"Unable to obtain Security Token Service from SSO '<vCenter fqdn>'","type":"com.vmware.evo.sddc.common.services.psc.exception.PscException"},{"message":"Server returned 'request expired' less than 0 seconds after request was issued, but it shouldn't have expired for at least 400


• From SDDC Manager's /var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log, below log snippets are found:

  YYYY-MM-DDTHH:MM:SS INFO  [common,##############,####] [c.v.e.s.i.s.PscInventoryServiceEmbeddedImpl,http-nio-127.0.0.1-####-exec-5] Returned PSCs: [Psc(domainId=########-####-####-####-############, vmName=<vCenter hostname>, vmManagementIpAddress=<vCenter IP Address>, vmHostname=<vCenter fqdn>, isReplica=false, port=443, ssoDomain=<SSO Domain>, subDomain=<SSO Domain>, sshHostKeyType=null, sshHostKey=null, bundleRepoDatastore=lcm-bundle-repo, datastoreName=<Datastore Name>, status=ACTIVE, version=<vCenter server version>, deploymentType=EMBEDDED)]
YYYY-MM-DDTHH:MM:SS INFO  [common,##############,####] [c.v.e.s.i.s.services.PscServiceImpl,http-nio-127.0.0.1-####-exec-5] Creating admin client using psc host <vCenter fqdn>
YYYY-MM-DDTHH:MM:SS WARN  [common,##############,####] [c.v.v.s.a.c.v.impl.AbstractClient,http-nio-127.0.0.1-####-exec-5] ******* WARNING ****** WARNING ****** WARNING *******
YYYY-MM-DDTHH:MM:SS WARN  [common,##############,####] [c.v.v.s.a.c.v.impl.AbstractClient,http-nio-127.0.0.1-####-exec-5] Possible remote API mismatch detected. Operation will continue, but errors are likely.
YYYY-MM-DDTHH:MM:SS WARN  [common,##############,####] [c.v.v.s.a.c.v.impl.AbstractClient,http-nio-127.0.0.1-####-exec-5] ******* WARNING ****** WARNING ****** WARNING *******
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.i.s.IdentityProviderServiceImpl,http-nio-127.0.0.1-####-exec-5] Failed to get the IDP type Identity Internal Server Error
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.i.r.a.c.v.IdentityProviderController,http-nio-127.0.0.1-####-exec-5] Unable to get sddc manager oidc information due to Identity Internal Server Error
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-####-exec-5] [######] SDDCMANAGER_GET_OIDC_INFO_FAILED
com.vmware.evo.sddc.identity.rest.api.error.IdentityRestServiceException: at com.vmware.evo.sddc.identity.rest.api.controller.v1.IdentityProviderController.getSddcWs1bOidcInfo(IdentityProviderController.java:444)
YYYY-MM-DDTHH:MM:SS ERROR [common,##############,####] [c.v.e.s.i.s.VcfServiceInventoryServiceImpl,http-nio-127.0.0.1-7100-exec-6] Error while trying to retrieve service http://127.0.0.1/sddc-manager-ui-app/about status, 500 Internal Server Error: "<html><EOL>  <head><EOL>    <title>Server failed to start !</title><EOL> <style type=

Cause

The connection between the Management vCenter server and the SDDC Manager is lost due to unexpected instances.

Resolution

• Check the connectivity between the Management vCenter server and the SDDC Manager.
• Reboot the Management vCenter server.
• And then restart SDDC Manager services using below command:
  
  /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh

Additional Information

In some instances, the vCenter server might have lost the IP address details or might have lost the NIC connection.

This issue also occurs if the NSX management console becomes inaccessible due to NSX edge node crashes.