vCenter Server vpxd fails to start after a reboot, patching, or certificate renewal

Products

• VMware vCenter Server

Issue/Introduction

• The vCenter Server vpxd service fails to start or initialize within the expected timeout period following a system reboot, update attempt, or certificate renewal.
• The web client is unusable. This symptom specifically impacts large-scale environments (greater than 500 managed ESXi hosts) utilizing Virtual Machine Encryption features, including encrypted VMDKs or Virtual Trusted Platform Modules (vTPM).
• During the startup sequence, vmon terminates the vpxd service because it is not marked as started within the configured timeout threshold. The following validation signature is observed in vCenter - /var/log/vmware/vpxd/vpxd.log file:
  
  info vpxd[PID] [Originator@6876 sub=vpxdvpxdSignal] Received a shutdown request, exiting

Environment

vCenter Server 8.0

Cause

During initialization, a high volume of encryption health checks (associated with VMDK encryption or vTPM) exhausts the available internal thread pool within the vpxd service. This thread exhaustion blocks standard vCenter Server operations and initialization routines, causing vpxd to exceed its startup timeout threshold and subsequently be terminated by vmon. In some cases vpxd may report to vmon as started but the web client still will not operate. 

Resolution

Engineering is aware of this condition and is working on a resolution. 

Workaround

1. Take an offline snapshot or file-based backup of the vCenter Server appliance before proceeding with database modifications.
   
   
2. SSH to the vCenter via root
   
   
3. Stop the vpxd service
   
   service-control --stop vpxd
   
   
4. Update the vpx_host table to force all managed ESXi hosts into a disconnected state
   
   /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "UPDATE vpx_host SET enabled = 0"
   
   Notes:
   
   • Modifying the enabled value from 1 to 0 instructs vCenter Server to bypass the host startup health checks, preventing thread exhaustion.
   • Hosts with lockdown mode enabled may not reconnect to vCenter using this script.
     
     
5. Start the vpxd service or manually restart all services to bring the vSphere Client online
   
   service-control --start vpxd
   
   or
   
   service-control --stop --all && service-control --start --all
   
   
6. Download the provided PowerCLI script attached to this article.
   
   
7. Execute the PowerCLI script (.\bulkreconnect.ps1) to systematically reconnect all ESXi hosts to the vCenter Server inventory in bulk. Any hosts which report as "Cannot complete login due to an incorrect user name or password" for their task will require a manual re-connect and validation.