Consolidating vCPUs for an SMP virtual machine can improve performance for some workloads

Products

• VMware vSphere ESXi

Issue/Introduction

ESX\ESXi CPU scheduler, by default, tries to place the vCPUs of an SMP virtual machine into as much Last-Level Cache (LLC) as possible. This is to maximize the cache capacity and memory bandwidth available to the virtual machine.

This policy might not be ideal for some workloads. In this case, the advanced setting vsmpConsolidate to TRUE instructs the ESX\ESXi scheduler to place the vCPUs of an SMP virtual machine into the fewest LLC’s possible. This policy better cache sharing for the virtual machines.

Environment

VMware ESX 4.0.x
VMware ESXi 4.0.x Embedded
VMware ESXi 4.0.x Installable
VMware vSphere ESXi 5.5
VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.0
VMware vSphere ESXi 5.1
VMware ESXi 4.1.x Installable

Resolution

If you think your SMP virtual machine might gain a performance benefit by having better cache sharing, enable the vSMP consolidate setting as follows:

Using the vSphere client:

1. Power off the virtual machine.
2. Right click the virtual machine and select Edit Settings.
3. Select the Options tab.
4. Under Advanced, click General, and on the right click the configuration Parameters button.
5. Click Add Row.
6. Add sched.cpu.vsmpConsolidate set to true.
7. Power on the virtual machine.

Note: This setting does not have any effect on current NUMA capable systems as the NUMA scheduler already tries to keep vCPUs NUMA and LLC local.