Virtual Machine rebooted with the following event: "The CPU has been disabled by the guest operating system"

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides steps to collect troubleshooting information that must be provided to the guest operating system vendor and to resolve this issue you must contact guest operating system vendor.

• A virtual machine restarts and reports the following event:
  
  The CPU has been disabled by the guest operating system. Power off or reset the virtual machine.
  
  
• ESXi - /vmfs/volumes/<datastorename>/<vmname>/vmware.log for the virtual machine shows the following messages:

[YYYY-MM-DDTHH:MM:SS] vcpu-0| I125: Vix: [vmxCommands.c:7191]: VMAutomation_HandleCLIHLTEvent. Do nothing.
[YYYY-MM-DDTHH:MM:SS] vcpu-0| I125: MsgHint: msg.monitorevent.halt
[YYYY-MM-DDTHH:MM:SS] vcpu-0| I125+ The CPU has been disabled by the guest operating system. Power off or reset the virtual machine.


Cause

The error message is seen when a guest operating system running in a VMware virtual machine intentionally halts the virtual CPU by executing the instructions CLI and HLT in succession. This can occur during a critical error or fault within the guest operating system.

This error can occur in:

• Windows virtual machines - Caused by the guest operating system crashing (BSOD).
• Linux virtual machines - Caused by a kernel panic or soft-lockups.

Resolution

This error requires analysis from the guest operating system vendor. It's important to gather relevant information below:

1. Note the virtual machine name and time of the outage. 
2. Collect the following logs as soon as possible after the outage:
   • Guest operating system logs (e.g. SoS report for RHEL). Engage the guest operating system vendor for more details on the correct logs to collect.
   • ESXi host logs. See Collecting diagnostic information for ESX/ESXi hosts and vCenter Server using the vSphere Web Client for more information.
3. If the VM did not restart and is hung on a panic screen (e.g. BSOD) collect the following:
   • Take a screenshot of the virtual machine console. 
   • In the inventory, Right Click on the VM, select 'Suspend' for the virtual machine, the checkpoint suspend (.vmss) and memory image (.vmem)  will be generated and can be found in the datastore from the virtual machine directory
   • Convert the checkpoint suspend files (.vmss and .vmem) from the virtual machine into a core dump file using the vmss2core utility. For more information, see the article Converting a snapshot file to memory dump using the vmss2core tool. 
4. Resume the virtual machine and reset the virtual machine.

If the guest operating system vendor identifies the crash is due to a VMware component (e.g. VMware Tools), contact VMware by Broadcom support. See Contact Broadcom support for more details.

Note: If using Linux, it is often recommended to disable soft lockups and NMI watchdog within the guest operating system to avoid spurious restarts in virtual environments. Ask the Guest Operating System vendor if disabling these settings would be applicable in the environment. e.g. RHEL - Chapter 11. Keeping kernel panic parameters disabled in virtualized environments / SUSE - What are all these "Bug: soft lockup" messages about?