Email alert "VMware Cloud Director cell with IP address ##.##.##.## is successful in reconnecting to the database" for one of the VMware Cloud Director cell

Products

• VMware Cloud Director

Issue/Introduction

• Frequent email alerts:
  
  VMware Cloud Director cell with IP address #### is successful in reconnecting to the database.


• The vmware-vcd-watchdog.log indicates the service is dead and attempting to restart.
• The request.log shows high frequency of API requests (e.g., >100/sec) resulting in HTTP/1.1 503 errors only on the cell which is reporting an alert. 

Environment

VMware Cloud Director: 10.6.x

Cause

The VCD cell thread pool (vcloud.http.maxThreads) becomes saturated by unoptimized API traffic. This is typically caused by automation or backup software hitting a specific cell IP directly rather than the Load Balancer, or by scripts that re-authenticate for every request instead of reusing session tokens. When the pool is saturated, the internal watchdog health-check fails, triggering a perceived database disconnection and service restart.

Resolution

To resolve this issue, ensure all automation and backup software is configured to use the environment's Load Balancer URL rather than a specific cell IP to allow for equitable load distribution to all the VCD cells.