VCF Operations UI in vSphere Client displays "no healthy upstream" after manual deletion of Ops virtual machines.

Products

• VMware vSphere Foundation
• VMware Cloud Foundation

Issue/Introduction

The VCF Operations UI in the vSphere Client continuously displays a "no healthy upstream" error if the existing VCF Operations virtual machines are manually deleted from the disk.

 

Normally, the deployment screen to reinstall VCF Operations should be displayed as shown below. However, due to the error, users are unable to access this screen.

Environment

VMware Cloud Foundation 9.0
VMware vSphere Foundation 9.0

Cause

This issue occurs because the VCF Operations virtual machines are deleted from the disk without unregistering their extensions from the vCenter.
The vCenter ExtensionManager retains the stale plugin registrations.

As a result, the vsphere-ui service attempts to connect to the non-existent backend using stale routing information, which results in the "no healthy upstream" error.

Resolution

If the VCF Installer cannot be used, you need to manually redeploy the VCF Operations appliance from the OVA to resolve this issue.

1. Deploy a new VCF Operations instance from the OVA.
   
   Note: You can reuse the exact same IP address and FQDN as the deleted nodes, or assign new ones.
   
   
2. Log in to the newly deployed VCF Operations UI.
   
   
3. Add and configure the vCenter adapter instance.
   
   
4. During the adapter configuration, a prompt will appear asking whether to add the vCenter for monitoring only, or for license/plugin management and monitoring purposes.
   Select the second option (license/plugin management and monitoring). The newly deployed VCF Operations will automatically take ownership of the vCenter license management and overwrite the stale plugin registrations in the vCenter Server.
   
   Note: For more details on each option, refer to Step 15 in the following documentation:
   Configuring a vCenter Account in VCF Operations