"no healthy upstream" error when attempting to access the vCenter Server vSphere Client

Products

• VMware vCenter Server

Issue/Introduction

• vCenter is down
• Logging in to the vCenter Server using the vSphere Client fails.
• The following error is observed when trying to access the vCenter Server via FQDN from a web browser:
  
  no healthy upstream
  
  
• In certain situations, you will get the "no healthy upstream" error immediately after clicking on "Launch vSphere Client":
  
  
• The error message is observed while redirecting to "websso" page:
  
  

Environment

• VMware vCenter server 7.x
• VMware vCenter server 8.x

Cause

This issue may occur due to a number of reasons, such as:

• The VPXD service on the vCenter Server is not running or not accepting connections due to:
  • Expired certificates
  • Disk space issues.
  • Running out of sessions (reached the maximum limit of 2000)
  • The service running out of memory
  • Scheduled maintenance, such as updates.
• The SSO service (vmware-stsd) is down or not responding due to:
  • The service running out of memory
  • Scheduled maintenance, etc.
• The file /etc/hosts has been modified from its default configuration, preventing services from starting.
• The Lookup service (lookupsvc) fails to start due to error received by LDAP client.

Resolution

Before troubleshooting, it is critical to understand what a "no healthy upstream" error means. "no healthy upstream" is an alternate error message for HTTP Return code 503, which indicates that the server is temporarily unable to handle the request.

• Note: Prior to making any changes on the vCenter Server, take a snapshot of the vCenter Virtual Machine.  If the vCenter Server is in linked mode with other vCenter Servers, refer to the article below for taking offline snapshots of each linked vCenter Server node.

VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice

1. To verify and renew/regenerate the self-signed certificates on a vCenter Server, utilize the vCert script following the article  vCert - Scripted vCenter expired certificate replacement.
2. To check for the LDAP issue referred to in the below article.  "LDAP Error Code 49"/Error (49) error in vmdird logs in vCenter Server