Navigating to Cluster and then Updates page on the vCenter Server shows "no healthy upstream"
search cancel

Navigating to Cluster and then Updates page on the vCenter Server shows "no healthy upstream"

book

Article ID: 405124

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • Navigating to Cluster > Update, Host > Update, and vCenter > Update shows an error mentioning "no healthy upstream"
  • Accessing the Lifecycle Manger from vCenter > Lifecycle Manager works without issues 
  • Logs on the vCenter Server have the following entries : 

    /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log mentions issues with session authentication and reattempts to login to the VUM server :
    [2025-07-22T11:17:42.292+08:00] [INFO ] nio-127.0.0.1-5090-exec-3167  com.vmware.vum.client.websocket.SpringConfigurator                Modify handshake for request: wss://<vcenter-fqdn>/ui/vum-ui/vum-websocket
    [2025-07-22T11:17:42.292+08:00] [INFO ] nio-127.0.0.1-5090-exec-3167  com.vmware.vum.client.websocket.VumWebSocketEndpoint              Websocket opened
    [2025-07-22T11:17:42.396+08:00] [WARN ] nio-127.0.0.1-5090-exec-3169  com.vmware.vim.vmomi.client.http.impl.ThumbprintTrustManager      Server is trusted; certificate validation by the configured VKS2 trustore succeeded while the thumbprint one failed
    [2025-07-22T11:17:42.403+08:00] [WARN ] nio-127.0.0.1-5090-exec-3169  com.vmware.vum.client.remoting.impl.VumServiceImpl                Request failed com.vmware.vim.binding.vim.fault.NotAuthenticated: The session is not authenticated.
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
            at java.lang.Class.newInstance(Class.java:442)
    .
    .
    .
    [2025-07-22T11:17:42.405+08:00] [WARN ] nio-127.0.0.1-5090-exec-3169  com.vmware.vum.client.remoting.impl.VumServiceImpl                Attempting relogin to VUM server
    [2025-07-22T11:17:42.469+08:00] [WARN ] nio-127.0.0.1-5090-exec-3169  com.vmware.vum.client.api.HostComplianceServiceImpl               (integrity.fault.NoEntities) {
       faultCause = null,
       faultMessage = null
    }

    /var/log/vmware/vsphere-ui/logs/apigw.log also has similar entries mentioning issues with authentication
    [2025-07-22T10:21:39.731+08:00] [INFO ] agw-httpclient-worker-2                             ApiGwApiProviderForwarder$RetryOnUnauthenticatedAsyncHandle [agw-0054650] Unauthenticated error for com.vmware.esx.hcl.compatibility_data.update$task by http://localhost:1080/esx-lcm . Will relogin and retry.
    [2025-07-22T10:21:39.732+08:00] [INFO ] agw-httpclient-worker-2                             VapiServiceAuthenticator [agw-0054650] Logging into http://localhost:1080/esx-lcm in domain vsphere.local(ccc33a09-d149-424f-8871-d442248c0659) with token ID _69db4020-69d1-4600-8102-4115b6404cab (opId=apigw:8ad06a08-b3c3-4971-83e9-540be91e7eb5:201004:6067)
    [2025-07-22T10:21:39.739+08:00] [INFO ] agw-httpclient-worker-8                             VapiServiceAuthenticator$LogoutAsyncHandle [agw-0054650] Session apigw-be-session-6006 at URI http://localhost:1080/esx-lcm is not even authenticated. No need to log out.
    [2025-07-22T10:21:39.745+08:00] [INFO ] agw-httpclient-worker-1                             VapiServiceAuthenticator$LoginAsyncHandle [agw-0054650] Logged into http://localhost:1080/esx-lcm. Back-end session 'apigw-be-session-6007'

Environment

.vCenter Server 8.x 

Cause

This can occur due to caching of sessions from vSphere UI. 

Resolution

  1. Utilize the vCert tool to verify for expired certificates within the STS or Machine certificate chain.
    Refer : vCert - expired certificate replacement script

  2. Use the lsdoctor tool to validate for SSL trust issues on the vCenter Server .
    Refer : Using the lsdoctor Tool

  3. If option 1 and 2 reveal no issues , restart the vSphere UI service using the command :
    service-control --stop vsphere-ui && service-control --start vsphere-ui