Unable to connect to the VMware Cloud Director portal due to database connection limits

Products

• VMware Cloud Director

Issue/Introduction

• Attempting to log into the VMware Cloud Director (VCD) Service Provider Admin or Tenant Portals fails.
• Attempting to log into VMware Cloud Director succeeds, but the closes ater login, kicking the user out.
• There are intermittent connectivity issues to the VMware Cloud Director cells.
• In the /opt/vmware/vcloud-director/logs/cell-runtime.log on the VMware Cloud Director cells, you see messages similar to:
  
  2024-10-16 06:44:42,552 | ERROR    | processor-ValFabric       | JDBCExceptionReporter          | [processor-ValFabric] Timeout: Pool empty. Unable to fetch a connection in 20 seconds, none available[size:75; busy:75; idle:0; lastwait:20000]. |

Environment

• VMware Cloud Director 10.5.x

Cause

This issue occurs because the database connection pool has been exhausted due to the cells reaching their continuous database connection limit.

Resolution

To resolve this issue there are the following options:
- The vmware-vcd service can be restarted on each cell as a temporary solution with the command: service vmware-vcd restart
- You can deploy additional Application cells into the VMware Cloud Director cluster up to the 6 cell limit.
- You can increase resources for your VMware Cloud Director cell VMs as this is a known issue if the appliances are undersized (from https://techdocs.broadcom.com/us/en/vmware-cis/cloud-director/vmware-cloud-director/10-5/vmware-cloud-director-installation-and-upgrade-guide-10-5/deployment-uprade-and-administration-of-the-vcd-appliance-install/deployment-and-initial-configuration-of-your-vcd-appliance-install/vcd-appliance-sizing-guidelines-install.html).

If you have already deployed the maximum number of cells in a VMware Cloud Director cluster, contact Broadcom Support for guidance on optimization of the database and note this Article ID (376122) in the problem description. For more information, see Creating and managing Broadcom support cases.

Additional Information

For more information on the maximum number of cells for a VMware Cloud Director cluster see VMware Cloud Director Appliance Sizing Guidelines.