Setting the CPU limit of virtual machines may impact the ESXi utilization on overcommitted systems

Products

• VMware vSphere ESXi

Issue/Introduction

Due to an issue in the ESXi scheduler, the overall utilization of the system might decrease slightly when a CPU limit is set on some virtual machines. Because of this unexpected drop in overall system utilization the usage of other virtual machines, which do not have CPU limit constraints, might be negatively impacted.

The issue only happens when:

1. The system is CPU overcommitted where the total number of VCPUs from all running virtual machines is greater than the total number of available PCPUs.
   
   
2. The system is heavily loaded and the total demand from all virtual machines, before setting a virtual machines CPU limit, exceeds at least 80% capacity of the system.

On under-committed or lightly-loaded systems, this issue does not happen.

Example symptom:

You have an 8 core ESXi system with no hyper-threading (i.e. the total system capacity is at 800%) and 16 virtual machines configured with 1 VCPU running on it. Each of those virtual machines is running a CPU-bounded workload. Hence, we have a 2x overcommitted system with total CPU demand close to 1600%. In this setup, each virtual machine will have roughly 50% CPU usage and the overall system utilization should be close to 100%.

If you then set a CPU limit on one of the single processor virtual machines to 10%, we expect the overall system utilization to remain close to 100%. The additional 40% freed CPU resource should be distributed evenly among the other 15 virtual machines, which results in a slight increase in their virtual machine usage. The increase of usage for other virtual machines is expected to be approximately equal to 40% / 15 = 2.67%.

However, due to this issue, the overall CPU utilization may drop slightly below 100% and the usage of other 15 virtual machines may not increase as expected.

Environment

VMware vSphere ESXi 6.0
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.5
VMware vSphere ESXi 5.0

Resolution

The issue is resolved in the:

• ESXi600-201507001 patch for ESXi 6.0. For more information, see VMware ESXi 6.0, Patch Release ESXi600-201507001 (2111982).
• ESXi550-201505002 patch for ESXi 5.5. For more information, see VMware ESXi 5.5, Patch Release ESXi550-201505002 (2114245).
• ESXi510-201503001 patch for ESXi 5.1. For more information, see VMware ESXi 5.1, Patch Release ESXi510-201503001 (2099286).

Currently, there is no workaround available.

Additional Information

For translated versions of this article, see:

• 日本語: オーバーコミットされたシステムで仮想マシンの CPU リソースの制限値を設定すると、ESXi ホストの CPU 使用率が低下する (2106460)