Virtual machine operation tasks fail on VMware Cloud Director - 'Max attempts to deploy on compute fabric exceeded configured value'

Products

• VMware Cloud Director

Issue/Introduction

• Virtual machine operation tasks fail on VMware Cloud Director - 'Max attempts to deploy on compute fabric exceeded configured value' 
• Errors seen on UI : 

Unable to perform this action. Contact your cloud administrator.

 

• Errors seen on vcloud-container-debug.log under /opt/vmware/vcloud-director/logs : 

 

com.vmware.ssdc.util.LMException: Internal Server Error
at com.vmware.ssdc.util.LMException.wrap(LMException.java:135)
at com.vmware.ssdc.library.ExceptionFactory.createFromMultiple(ExceptionFactory.java:32)
at com.vmware.ssdc.backend.DeployVAppVmSubsetActivity.checkFuturesForException(DeployVAppVmSubsetActivity.java:529)
at com.vmware.ssdc.backend.DeployVAppVmSubsetActivity$DeployNetworksPhase.invoke(DeployVAppVmSubsetActivity.java:210)
at com.vmware.vcloud.activity.executors.ActivityRunner.runPhase(ActivityRunner.java:175)
at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:112)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: com.vmware.vcloud.api.presentation.service.InternalServerErrorException: Internal Server Error
at com.vmware.vcloud.common.future.FutureUtil.convertExecutionException(FutureUtil.java:264)
at com.vmware.vcloud.common.future.FutureUtil.checkCompletedFuture(FutureUtil.java:203)
at com.vmware.ssdc.backend.DeployVAppVmSubsetActivity.checkFuturesForException(DeployVAppVmSubsetActivity.java:522)
... 8 more

Caused by: java.lang.Exception: Max attempts to deploy on compute fabric exceeded configured value
        at com.vmware.ssdc.backend.DeployVmOnFabricsActivity$DeployVMsComputeFabricPhase.invoke(DeployVmOnFabricsActivity.java:190)
        at com.vmware.vcloud.activity.executors.ActivityRunner.runPhase(ActivityRunner.java:175)
        at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:112)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)


com.vmware.ssdc.library.exceptions.InsufficientMemoryResourcesException: The operation could not be performed, because there are insufficient memory resources.

or, 

com.vmware.ssdc.library.exceptions.InsufficientCpuResourcesException: The operation could not be performed, because there are insufficient CPU resources.

Environment

VMware Cloud Director 10.x

Cause

This happens due to Memory or CPU resource overcommitment for the Organization VDC. 

Resolution

To fix the issue, check resource settings under Provider > Resources > Organization VDC > VDC name > Settings > Allocation and check reservation settings for CPU (CPU resources guaranteed) or, Memory (Memory resources guaranteed) and then Edit and reduce the reservation to a lower value.