ERROR "New value for CPU resource is less than the current usage by all VMs" while modifying the CPU quota from the allocated size on an Organization VDC.

Products

• VMware Cloud Director

Issue/Introduction

• When modifying the CPU quota of an Organization VDC in VMware Cloud Director, the following error message is encountered:

  "Unable to update the Organization VDC configuration because the new value for CPU resource is less than current usage by all Virtual Machines within this Organization VDC. - ValidationException VRP_NEWLY_CONFIGURED_CPU_RESOURCE_INSUFFICIENT".

  
  

• Despite the OrgVDC having sufficient CPU resources at the vCenter level, the change still fails.
  
  Example:

Environment

VMware Cloud Director 10.5 and Later

Cause

If the vCPU speed for the OrgVDC was modified after its initial configuration, the CPU limits assigned to existing VMs do not update automatically. As a result, some VMs retain higher CPU allocations, preventing the quota reduction.

Resolution

1. Verify CPU Usage in vCenter:

   • Compare the CPU allocation displayed in VCD with actual usage in vCenter.

   • Ensure that the total usage in vCenter does not exceed the new CPU quota.

2. Check vCD Logs for Errors:

   • SSH into the VCD cells and navigate to the logs folder:

     cd /opt/vmware/vcloud-director/logs

   • Search for errors related to the job using its JOB ID:

     grep -il "JOBID"

   • The file “vcloud-container-debug.log” had an entry with the JOB ID. Look for messages indicating CPU resource validation failure.

     ERROR  | compute-fabric-activity-pool71 | GenericVirtualResourcePool   | Current usage of 12,000Mhz cpu is more than the newlyconfigured cpu 6,000Mhz. | requestId=########-####-####-############,request=PUT https://<vcdfqdn>.com/api/admin/vdc/########-####-####-############

      

3. Identify Virtual Machines Causing the Issue:

   • Type 'db' to access the vcloud database.
     

     sudo -i -u postgres psql vcloud

      

   • Retrieve the vrp_id of the OrgVDC:

     SELECT * FROM vrp WHERE name LIKE '%OrgName%';

     Example:
      vcloud=# SELECT * FROM vrp where name like '%name-OVDC%';
     -[ RECORD 1 ]-----------+----------------------------------------------------------
     id                      | ########-####-####-############
     name                    | 'OVDC Name'
     description             |
     computehub_set_id       | ########-####-####-############
     creation_status         | READY
     allocation_model        | 0
     overprov_factor_cpu     | 0.2000
     overprov_factor_mem     | 0.2000
     vcpu_mhz                | 1000
     memory_allocation_mb    | 0
     memory_limit_mb         | 16384
     cpu_allocation_mhz      | 0
     cpu_limit_mhz           | 12000
     version_number          | 33
     upgrade_status          | 0
     is_elastic              | t
     include_memory_overhead | f

      

   • Identify VMs with outdated CPU limits:

     SELECT name, moref, vcpu_count, cpu_limit, (vcpu_count * <vcpu_mhz value from the previous query>) as vCPU FROM vm_inv WHERE moref IN (SELECT vmmoref FROM computevm WHERE deployment_status IN ('DEPLOYED', 'PENDING_DEPLOYMENT') AND vrp_id = <Output of ID from the previous query> );

     Example:
            name        |  moref   | vcpu_count | cpu_limit | vcpu
     -------------------+----------+------------+-----------+------
          vm-name       | vm-##### |          2 |      4000 | 2000
          vm-name       | vm-####  |          4 |      8000 | 4000
     (2 rows)
     
     ====================

      

   • If VMs have a higher cpu_limit than expected, Kindly reboot the affected VMs.
     
     

4. Retry Reducing the CPU Quota:

   • After rebooting the affected VMs, the current CPU usage should now reflect the updated vCPU speed in OVDC.

   • Attempt to reduce the CPU quota should complete successfully.

Additional Information

Aria operations monitoring can be leveraged to determine when the vCPU Speed changes were made in OVDC.

Note: From the screenshot it shows that the changes to 'vCPU speed' in OVDC was made in November 2024, However the changes did not take effect as the VMs were not rebooted.