vNUMA is disabled if vCPU hotplug is enabled

Products

• VMware vSphere ESXi

Issue/Introduction

When selecting "Enable CPU Hot Add" in the virtual machines Edit Settings dialogue (vmx option: "vcpu.hotadd = true"), vNUMA is disabled for the VM. As a result, even if the virtual machine is created with more vCPUs than cores in one of the underlying hardware's NUMA node (or threads if "numa.vcpu.preferHT = true" is set), only one virtual node (VPD) will be presented to the VM and the memory interleaved over the multiple resulting NUMA clients (PPDs).


You might see the following in the vmware.log on ESXi 6.7 and later:

<datetime> vmx| I125: numa: Hot add is enabled and vNUMA hot add is disabled, forcing UMA.

Alternatively in ESXi 6.5 and earlier:

<datetime> vmx| W110: NUMA and VCPU hot add are incompatible. Forcing UMA.

Environment

VMware vSphere ESXi 6.X
VMware vSphere ESXi 7.x

Resolution

None. If you do not plan to use vCPU hotplug, do not enable it. Add the maximum vCPUs that might be needed by the workload. In rare cases, where vCPU hot plug is required, try to limit the maximum number of vCPUs to stay below the physical NUMA node core count so that the virtual machine can be scheduled in a single NUMA node.

Steps to Enable/ Disable CPU Hot Add:

• Login to vCenter UI
• Locate the VM and Power it off.
• Right click on the Virtual Machine (centos8) > Edit Settings > Virtual Hardware > expand CPU > CPU Hot Plug.
• To disable CPU hot Plug: Uncheck the box "Enable CPU Hot Add".
• To enable CPU hot Plug: Check the box "Enable CPU Hot Add". 

Additional Information

Using Virtual NUMA (VMware Docs)

Enabling vCPU HotAdd creates fake NUMA nodes on Windows (broadcom.com)