The error “[500] An error occurred while fetching identity providers” appears while login to vCenter Server UI.

Products

• VMware vCenter Server 7.0

Issue/Introduction

vCenter Server log in fails with the error: "[500] An error occurred while fetching identity providers. Try again. If problem persists, contact your administrator."

The following log entries are found in: /var/log/vmware/vsphere_ui/logs/vsphere_client_virgo.log

YYYY-MM-DD HH:MM:SS [WARN ] http-nio-5090-exec-9 70000004 100004 ###### c.v.vsphere.client.security.oauth2.logout.LogoutRequestHandler Unable to determine the identity provider type. Logout request will be skipped.
YYYY-MM-DD HH:MM:SS [INFO ] http-nio-5090-exec-4 70000005 100004 ###### com.vmware.vsphere.client.security.oauth2.LoginRequestHandler Received Multi login request
YYYY-MM-DD HH:MM:SS [INFO ] http-nio-5090-exec-4 70000005 100004 ###### com.vmware.vise.vim.vapi.StaticEndpointVapiConnectionManager Connected to vAPI endpoint https://vcenter.example.org:443/site/api
YYYY-MM-DD HH:MM:SS [ERROR] VapiAsyncCall-101 com.vmware.vise.vim.vapi.DefaultVapiConnectionControl Maximum number of attempts reached while trying to call com.vmware.vcenter.identity.providers.list
YYYY-MM-DD HH:MM:SS [ERROR] http-nio-5090-exec-4 70000005 100004 ###### com.vmware.vsphere.client.security.oauth2.LoginRequestHandler An error occurred while fetching providers com.vmware.vapi.std.errors.Unauthenticated: Unauthenticated (com.vmware.vapi.std.errors.unauthenticated) => {
    messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
    id = vapi.method.authentication.required,
    defaultMessage = This method requires authentication.,
    args = [],
    params = <null>,
    localized = <null>
}],
    data = <null>,
    errorType = UNAUTHENTICATED,
    challenge = <null>
}
    at java.lang.Thread.getStackTrace(Thread.java:1559)

/var/log/vmware/trustmanagement/trustmanagement-svcs.log:

YYYY-MM-DD HH:MM:SS [tomcat-exec-14 INFO com.vmware.identity.token.impl.SamlTokenImpl opId=] SAML token for SubjectNameId [value=machine-<machineID>@vsphere.local, format=http://schemas.xmlsoap.org/claims/UPN] successfully parsed from XML
YYYY-MM-DD HH:MM:SS [tomcat-exec-14 INFO com.vmware.identity.token.impl.X509TrustChainKeySelector opId=] **Failed to find trusted path to signing certificate** <STS Certificate Subject, example - C=US,CN=ssoserverSign\,dc\=vsphere\,dc\=local>
java.security.cert.CertPathBuilderException: **Unable to find certificate chain.**
    at org.bouncycastle.jcajce.provider.PKIXCertPathBuilderSpi.engineBuild(Unknown Source)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at com.vmware.identity.token.impl.X509TrustChainKeySelector.verifyTrustedPathExists(X509TrustChainKeySelector.java:197)
    at com.vmware.identity.token.impl.X509TrustChainKeySelector.select(X509TrustChainKeySelector.java:116)

This issue can also occur if the vCenter Machine_SSL certificate, Solution User certificates, or the STS signing certificate have expired.

To identify which certificate has expired, you can use the vCert Script:

• Download and install vCert on the vCenter Server Appliance following the steps in the Installation section.
• Select Option 1 to check the current certificate status.
• Select Option 2 to view the certificates:
  • Select Option 1 to verify the Machine_SSL certificate.
  • Select Option 2 to verify the Solution User certificates.
  • Select Option 8 to verify the STS signing certificates.

Environment

vCenter Server 8.x
vCenter Server 7.x

Cause

The issue occurs due to expired certificates in the vCenter Server. This can include the Machine_SSL certificate, Solution User certificates, and the STS (Security Token Service) signing certificate. These certificates are essential for authentication and secure communication between vCenter services. Once expired, internal services such as STS and SSO (Single Sign-On) cannot authenticate properly, leading to the "[500] An error occurred while fetching identity providers" error on the vCenter UI.

Resolution

Verification of Certificate Expiration Dates:

Before attempting to fix, you can quickly verify what certificates are expired with the following command:

for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done

a. Resetting the STS Certificate

Follow the steps below to reset the STS certificate:

Note: Perform these steps only if the following error messages are observed in /var/log/vmware/trustmanagement/trustmanagement-svcs.log:

• Failed to find trusted path to signing certificate
• Unable to find certificate chain

• Take a snapshot before proceeding with the replacement:
  • Take a no memory snapshot of the vCenter Server if it is in standalone mode.
  • If in linked mode, take powered-off snapshots of all vCenter Servers in the same SSO domain.
• Use the vCert utility to perform the replacement of the certificate:
  Download and install vCert on the vCenter Server Appliance as described in the Installation section.
• Check STS signing certificate:
  From the Menu 2: View Certificate Info, select Option 8 - View STS Signing Certificates.
• Replace STS signing certificate:
  From the Menu 3: Manage Certificates, select Option 8 - STS Signing Certificates to initiate the replacement.

For more details on resetting the STS certificate, refer to the KB article: "Signing certificate is not valid" error in vCenter Server Appliance.

b. Replacing expired Machine SSL or Solution User certificates

If the Machine SSL or Solution User certificates have expired, follow the steps below:

1. From the main menu, select Option 3 - Manage Certificate.
2. To replace the Machine SSL certificate, select Option 1 - Machine SSL Certificate.
3. To replace the Solution User certificate, select Option 2 - Solution User Certificates.