How to scale up Cloud Director Availability 4.x for large scale environments

Products

• VMware Cloud Director

Issue/Introduction

This article provides information on how to scale up a Cloud Director Availability deployment in a large scale cloud environment, where Cloud Director Availability is managing over 6000 replications.

Environment

VMware Cloud Director Availability 4.x

Resolution

In the large scale environment it is recommended to perform the following steps in order to avoid out of memory errors on the Cloud Director Availability appliances.

Note: The following process only needs to be applied to the Cloud Replication Management, Replicator, and Tunnel appliances on a cloud site. No actions are required on On-Premises appliances.

Increase the memory of the Cloud Replication Management appliance

1. Log into the vSphere Client where you can manage the Cloud Director Availability appliances.
2. Power off the Cloud Replication Management appliance.
3. Right-click on the appliance and click Edit Settings.
4. In the Virtual Hardware section, increase the CPUs from 2 cores to 4 cores.
5. In the Virtual Hardware section, increase the memory from 4GB to 12GB.
6. Click OK.
7. Power on the Cloud Replication Management appliance.

Configure the Cloud Director Availability appliances

Cloud service

1. SSH to the Cloud Replication Management appliance and log in as root.
2. Navigate to the Cloud Service configuration directory:

cd /opt/vmware/h4/cloud/config

3. Take a backup of the application.properties file:

cp application.properties application.properties.bak

4. Append the application.properties file with the following lines:

spring.jpa.properties.hibernate.query.plan_cache_max_size=256
spring.jpa.properties.hibernate.query.plan_parameter_metadata_max_size=64
spring.jpa.properties.hibernate.query.in_clause_parameter_padding=true

5. (Optional) If you plan to start more than 100 concurrent protections, uncomment the jobengine.threads.per.user line and double the value:

# Maximum number of allowed threads per user
jobengine.threads.per.user=20

Note: From Cloud Director Availability 4.7 onwards, you no longer need to increase the maximum number of allowed threads per user when you want to start a large number of concurrent protections. If you are using Cloud Director Availability 4.7 or later, skip step 5 and proceed directly to step 6.

6. (Optional) If you plan to replicate vApp Templates and leverage automatic vApp Template protection, uncomment the vcd.vapp.settings.timeout line and double the value:

vcd.vapp.settings.timeout=600000

7. Restart the Cloud service for the changes to take effect:

systemctl restart cloud

Manager service

1. SSH to the Cloud Replication Management appliance and log in as root.
2. Navigate to the Manager Service configuration directory:

cd /opt/vmware/h4/manager/config

3. Take a backup of the application.properties file:

cp application.properties application.properties.bak

4. Append the application.properties file with the following lines:

spring.jpa.properties.hibernate.query.plan_cache_max_size=256
spring.jpa.properties.hibernate.query.plan_parameter_metadata_max_size=64
spring.jpa.properties.hibernate.query.in_clause_parameter_padding=true

5. (Optional) If you plan to start more than 100 concurrent protections, uncomment the jobengine.threads.per.user line and double the value:

# Maximum number of allowed threads per user
jobengine.threads.per.user=20

Note: From Cloud Director Availability 4.7 onwards, you no longer need to increase the maximum number of allowed threads per user when you want to start a large number of concurrent protections. If you are using Cloud Director Availability 4.7 or later, skip step 5 and proceed directly to step 6.

6. Append the application.properties file with the following lines:

sync.scheduler.thread.count=100
health.scheduler.thread.count=100

7. Restart the Manager service for the changes to take effect:

systemctl restart manager

Replicator service

1. SSH to the Replicator appliance and log in as root.
2. Navigate to the Replicator Service configuration directory:

cd /opt/vmware/h4/replicator/config

3. Take a backup of the application.properties file:

cp application.properties application.properties.bak

4. Append the application.properties file with the following lines:

spring.jpa.properties.hibernate.query.plan_cache_max_size=256
spring.jpa.properties.hibernate.query.plan_parameter_metadata_max_size=64
spring.jpa.properties.hibernate.query.in_clause_parameter_padding=true

5. (Optional) If you plan to start more than 100 concurrent protections, uncomment the jobengine.threads.per.user line and double the value:

# Maximum number of allowed threads per user
jobengine.threads.per.user=20

​​​​​​​Note: From Cloud Director Availability 4.7 onwards, you no longer need to increase the maximum number of allowed threads per user when you want to start a large number of concurrent protections. If you are using Cloud Director Availability 4.7 or later, skip step 5 and proceed directly to step 6.

6. Restart the Replicator service for the changes to take effect:

systemctl restart replicator

Tunnel service

1. SSH to the Tunnel appliance and log in as root.
2. Navigate to the systemd services directory:

cd /lib/systemd/system

3. Update the LimitNOFILE line in the tunnel.service file:

LimitNOFILE=40960

4. Restart the Tunnel service for the changes to take effect:

systemctl restart tunnel

Additional Information

For more information on the scale limit for active protections per Replicator instance, see VMware Configuration Maximums.