Error : "Total memory required for all VMs in the VRP including current VM(s) being deployed [##,###,###MB] is greater than VRP maxResourceValue ##,###,###MB." when powering on or deploying vApp/VM(s) in VMware Cloud Director

Products

• VMware Cloud Director

Issue/Introduction

• Powering ON or deploying vApps from VMware Cloud Director(VCD) UI is failing with the error: 
  Total memory required for all VMs in the VRP including current VM(s) being deployed [##,###,###MB] is greater than VRP maxResourceValue ##,###,###MB.
• Only one Organization VDC(OVDC) is affected. 
• Powering ON the same VMs from vCenter(VC) works as expected.
• VCD UI shows there is enough memory resources available for the vApps or VMs to power on.

• In /opt/vmware/vcloud-director/logs/vcloud-container-debug.log, there are entries of the following type:

  com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.performAdmissionControl(GenericVirtualResourcePool.java:2840)
ERROR    | Backend-activity-pool-4016 | FutureUtil                     | Failed to deploy VM fabrics, VM id: fabricFuture | ... com.vmware.vcloud.fabric.compute.ValidationException: Admission control failed for VRP:[name: <OVDC Name> (########-####-####-####-############), id: ########-####-####-####-############] during the deployment of VMs:[[[vcId=########-####-####-####-############, moref=vm-########]]]. Total memory required for all VMs in the VRP including current VM(s) being deployed [##,###,###MB] is greater than VRP maxResourceValue ##,###,###MB.
        at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.verifyResourceConstraint(GenericVirtualResourcePool.java:3036
 
ERROR    | Backend-activity-pool-4015 | GenericVirtualResourcePool     | Admission control failed for VRP:[name: <OVDC Name> (########-####-####-####-############), id: ########-####-####-####-############] during deployment of VMs:[[[vcId=########-####-####-####-############, moref=vm-########]]]. Total memory required for all VMs in the VRP including current VM(s) being deployed [##,###,###MB] is greater than VRP maxResourceValue ##,###,###MB. | ...

Environment

VMware Cloud Director 10.6.1

Cause

This issue can occur if the memory limit on one or many of the VMs in the affected OVDC is incorrectly set/updated from the VC. This results in the total memory consumption of the OVDC to be greater than what is set on the OVDC itself.

NOTE: VCD objects should only be managed from VCD and not from VC directly.

Resolution

To resolve the issue, the memory limit set on the VM directly in vCenter must be removed. 

Identify the VM contributing to the incorrect resource calculation from the VCD DB as per the steps below:

1. Note the name of the OVDC which is affected.
2. Log in via web console or by using an SSH client to the primary appliance console as root.
3. Connect to the vcloud database with:
   sudo -u postgres psql vcloud
   (or) 
   db 
4. Using the name identified from step 1, populate the following script where highlighted with the OVDC name and execute it.
   
   select
vvm.name,
vminv.name,
cvm.vmmoref,
cvm.id,
cvm.deployment_status,
vminv.mem_reservation,
vminv.mem_limit,
cvm.memory_configured_mb,
coalesce(vminv.mem_limit, cvm.memory_configured_mb) as co,
(vminv.mem_overhead)/1048576 as combined
from computevm cvm, vm_inv vminv, vrp_rp vrprp, vapp_vm vvm 
where cvm.vrp_id in (select id from vrp where name like '%<OVDC name%') and 
cvm.vmmoref=vminv.moref and cvm.vc_id=vminv.vc_id and 
cvm.id=vvm.cvm_id and cvm.vc_id=vrprp.sub_rp_vc_id and 
vminv.vc_id=vrprp.sub_rp_vc_id and 
vrprp.vrp_id=cvm.vrp_id and 
vminv.resource_pool_moref=vrprp.sub_rp_moref and 
(cvm.deployment_status='PENDING_DEPLOYMENT' or cvm.deployment_status='DEPLOYED')
and vminv.mem_limit != -1
and vminv.mem_limit != cvm.memory_configured_mb;
   
   
5. Remove the incorrect limit set on the virtual machine with the steps below:
   1. Log in to vCenter Server using the web client interface. 
   2. Navigate to the inventory and find the specific VM we identified in Step 4.
   3. Power off the VM .
   4. Right-click the VM and select "Edit Settings". 
   5. In the "Edit Settings" window, locate and expand the "Memory" section. 
   6. Adjust Memory Allocation to reflect the correct values set from the VCD UI.
6. From VCD UI, verify the power ON of the VMs are successful.