Virtual Machine becomes unresponsive or hangs when the CPU Limit is set to 0

Products

• VMware vSphere ESXi

Issue/Introduction

A virtual machine hangs or becomes unresponsive.

/vmfs/volumes/datastore_uuid/VM_folder/vmware.log for the VM logs a change to "sched.cpu.max"

YYYY-MM-DDTHH:mm:ss | vmx| I005: VigorTransportProcessClientPayload: opID=xxxxxx seq=62963: Receiving Sched.SetCpuMax request.
YYYY-MM-DDTHH:mm:ss | vmx| A000: ConfigDB: Unsetting "sched.cpu.max"
YYYY-MM-DDTHH:mm:ss | vmx| I005: VigorTransport_ServerSendResponse opID=xxxxxx seq=62963: Completed Sched request.

A short while later, /vmfs/volumes/datastore_uuid/VM_folder/vmware.log contains entries indicating that there is no response from VMware Tools within the guest OS:
YYYY-MM-DDTHH:mm:ss | vmx| I005: GuestRpcSendTimedOut: message to toolbox timed out.
YYYY-MM-DDTHH:mm:ss | vmx| I005: Tools: [AppStatus] Last heartbeat value 103716 (last received 164s ago)
YYYY-MM-DDTHH:mm:ss | vmx| I005: TOOLS: appName=toolbox, oldStatus=1, status=2, guestInitiated=0.
YYYY-MM-DDTHH:mm:ss | vmx| I005: GuestRpcSendTimedOut: message to toolbox timed out.

 

/var/run/log/hostd on the ESXi host contains logging similar to the following showing that the "CPU max" was set to 0:
YYYY-MM-DDTHH:mm:ss  verbose hostd[8817032] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore_uuid/VM_folder/vm_name.vmx opID=xxxxxx user=vpxuser] Preparing resource pool update
YYYY-MM-DDTHH:mm:ss  info hostd[8817032] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore_uuid/VM_folder/vm_name.vmx opID=xxxxxx user=vpxuser] State Transition (VM_STATE_ON -> VM_STATE_RECONFIGURING)
YYYY-MM-DDTHH:mm:ss  info hostd[2100248] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 5288601 : Issue detected on ESXi_hostname in ha-datacenter: CpuSched: 2729: Container group(xxxxxx): host/user/vm.xxxxxx is configured with 0 CPU max

Environment

vCenter 7.0

vCenter 8.0

ESXi 7.0

ESXi 8.0

Cause

This is caused by a user or product setting a CPU Limit of 0 on a VM.

Setting the CPU Limit to 0 starves the VM of CPU resources, causing it to hang.

Resolution

To resolve this issue, remove or increase the CPU Limit on the VM. For further details, see Configure Your Resource Allocation Settings in vSphere

To prevent user error, from vCenter 7.0U3 it is no longer possible to set a CPU Limit of 0 via the vSphere Client.

However, it is still possible to make this change via API. Any automation or external products that are performing tasks against VMs at the time should be checked to see which one made the change.