Multiple VM Power on operation freeze at 65% and vCenter UI is unresponsive.

Products

• VMware vCenter Server 8.0

Issue/Introduction

Powering on VMs causes vCenter to become unresponsive, resulting in extremely slow response times and rendering the vCenter server non-functional.

Symptoms:

vSphere UI banner for partner vCenter server shows alet banner as "Could not connect"

/var/log/vmware/vsm/vsm.log
Logs showing connectivity issues between the vCenter and vService Manager (VSM), with error messages like:

ERROR [Thread-4] VCenterListener.java 524 - com.vmware.vim.vmomi.client.exception.SslException: Server certificate chain is not trusted and thumbprint verification is not configured

Environment

VMware vCenter Server 8.x

Cause

The root cause of the issue was related to the vSphere Service Manager (VSM) rejecting API calls from vpxd due to mismatched certificate chains. Specifically, the custom Machine SSL certificate was not present in the vCenter's trusted root store, which prevented the successful completion of power-on operations

Resolution

To resolve the issue, follow the below-mentioned steps:

1. Download and Install Intermediate + Root Certificates:
   
   
• The necessary intermediate and root certificates were downloaded (via browser) and added to the TRUSTED_ROOTS keystore through the vCenter GUI:
• Navigate to Administration → Certificate → Certificate Management.
• Add the downloaded intermediate and root certificates.
2. Restart vpxd and VSM Services:
3. Confirm Operation:
   
   • Navigate to Administration → Certificate → Certificate Management →TrustedRoot