Invalid vCenter Status: All required services are not up! Stopped services: 'hvc'.

Products

• VMware vCenter Server

Issue/Introduction

• Running backup using vCenter Appliance Management Interface (VAMI) results into an error: Invalid vCenter Server Status: All required services are not up! Stopped services: 'hvc'. 

• Hybrid vCenter Service (HVC) is in a stopped state.
• Check solution user certificate expiration for the 'hvc' service using command in vCenter server SSH session:

# for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;

Environment

vCenter Server 8.x

Cause

In VMware Endpoint Certificate Store (VECS), Certificate associated with Store: hvc is expired

Resolution

To restore the HVC service and resume backup operations, the expired Solution User certificates must be replaced using below steps:

 

Note: Take vCenter server snapshot. (In case if the vCenter servers are part of Enhanced Linked Mode, make sure to take offline snapshots of each vCenter server)

 

1. Log into the vCenter as root via SSH.

2. Replace Solution User certificates using the steps given in this KB article: How to replace the vCenter Server Solution User certificates with VMCA issued certificate

Note: This process will trigger a restart of vCenter services

3. Once the services have restarted, verify the HVC service status (service-control --status vmware-hvc) and initiate a manual backup via the VAMI.