Cisco Intersight plugin fails to load or shows error: 'no healthy upstream servers' in vCenter

Products

• VMware vCenter Server

Issue/Introduction

• The Cisco Intersight plugin displays a "no healthy upstream servers" error in the vSphere Client.
• The plugin (CIVP) is missing from the vCenter Managed Object Browser (MOB).
• The vSphere Lifecycle Manager (vLCM) is unable to retrieve the latest driver or firmware add-ons when configuring custom images.
• Incomplete or stale plugin registration is observed in the Serenity DB/UI cache.

Environment

VMware vCenter Server

Cause

This issue is typically caused by stale or incomplete plugin registration within the vCenter Server. This may occur if the plugin registration in the Managed Object Browser (MOB) does not match the cached data in the vSphere UI Serenity database, or if the user account associated with the plugin lacks sufficient permissions (e.g., Read-Only instead of Administrator) to complete the installation and backend connection.

Resolution

Follow these steps to perform a teardown and purge the plugin cache to allow for a clean re-registration:

Prerequisites

1. Take a snapshot of the vCenter Server Appliance (VCSA) before proceeding.
2. Ensure you have the credentials for a user with Administrator privileges for the plugin registration.

Procedure

1. Unregister the Stale Plugin:
   • Navigate to the vCenter MOB at https://<vcenter_fqdn>/mob/?moid=ExtensionManager.
   • Locate the Cisco Intersight plugin ID (typically com.ucs.manager.plugin CIVP or similar).
   • Use the UnregisterExtension method to remove the stale entry.
   • Clear the vSphere UI Cache:
2. SSH into the vCenter Server Appliance as root and stop the vSphere UI service:
   • service-control --stop vsphere-ui
3. Navigate to the Serenity DB/UI cache directory:
   • cd /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity
4. Remove the directory associated with the Cisco plugin:
   • rm -rf <plugin_folder_name>
5. Restart the vSphere UI service:
   • service-control --start vsphere-ui
6. Re-enable the Plugin in Cisco Intersight/UCS:
   • Log in to the Cisco Intersight or UCS Manager console.
   • Navigate to the plugin settings for the affected vCenter.
   • Disable the plugin, wait for the task to complete, and then Enable it again to trigger a fresh registration.

NOTE: Ensure the service account used for the vCenter target in Intersight has Administrator level privileges to ensure the plugin can register and communicate with the backend services.

Additional Information

• Cisco Intersight Plugin for VMware vSphere