Workload domain expansion fails with the error "Validation Error: Not enough resources available for workload expansion"

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• Workload domain expansion fails with the error "Validation Error: Not enough resources available for workload expansion"
• The workload domain being expanded has a redundancy level of "High".
• The only available free hosts to be used in the expansion are in a different rack from the hosts that already exist in the workload domain.
• You see output similar to the following after querying the https://localhost:8443/vrm-ui/api/1.0/iaas/max-expansion-capacity/<domain id> API:

{"status":"Success","code":"VRM080800I","message":null,"object":{"cpu":0.0,"cpuUnit":"GHZ","core":0,"coreUnit":"CORES","memory":0.0,"memoryUnit":"GB","storage":0.0,"storageUnit":"TB"}}

Note: The preceding excerpt is only an example. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware Cloud Foundation 2.x

Cause

This issue is caused when the expansion is attempted on a Workload domain whose redundancy is set to "High". In this scenario, only the free hosts available in the same rack as the workload domain would be considered for expansion. Workload domain expansion across the racks is not allowed in this scenario.

Resolution

This is a known issue affecting VMware Cloud Foundation 2.x. There is currently no resolution.

Workaround:
Either of the below workarounds can be used to complete the workload domain expansion.

1. Add free hosts to the same rack where the workload domain is created for expansion.
2. Reduce the redundancy on the workload domain considered for expansion to "Normal" by following the procedure below.

1. SSH to the SDDC Manager Controller VM.
2. Issue the following command to obtain the domain id value for the workload domain being expanded:

/opt/vmware/cassandra/apache-cassandra-2.2.4/bin/cqlsh -e "select id,name,redundancy from vrmkeyspace.domain"
 

Note: You will see output similar to the following:

 id                                   | name        | redundancy
--------------------------------------+-------------+------------
 c4ace1b6-####-####-####-########60a | vi-workload |       HIGH
 064debb0-####-####-####-########726 |        MGMT |     NORMAL

Note: Make a note of the id value for the workload domain being expanded. 

3. Issue a command similar to the following command to set workload domain redundancy to NORMAL:

/opt/vmware/cassandra/apache-cassandra-2.2.4/bin/cqlsh -e "update vrmkeyspace.domain set redundancy='NORMAL' where id=c4ace1b6-72dd-4894-be33-c0ebc4b4b60a"

     
           Note: Replace c4ace1b6-####-####-####-########60a with the domain id of the workload domain obtained in Step 2b.

Note: The workload domain expansion operation should complete successfully at this point.