Troubleshooting virtual machine power on task hangs

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

This article provides steps to troubleshoot a virtual machine (VM) that hangs when powering on.

Symptoms:
When powering on a virtual machine (VM) the power on task does not complete.
VM options may appear greyed out.

Environment

VMware vCenter Server Appliance 6.5.x
VMware vCenter Server 6.7.x
VMware vCenter Server Appliance 6.7.x
VMware vSphere ESXi 6.7
VMware vSphere ESXi 7.0.0
VMware vCenter Server 7.0.x
VMware vCenter Server 6.5.x
VMware vSphere ESXi 6.5

Cause

This issue can be caused by:

• ESXi OS problems
• ESXi/vCenter OS waiting for administrator answer to question
• Incompatible ESXi CPU for snapshot
• Pending tasks
• Storage array/datastore problems
• VM configuration problems
• VM guest OS incompatibility with ESXi host
• vCenter OS problems

Resolution

• For powered on VM's that are failing to load the gust operating system (OS), see Troubleshooting unresponsive guest operating system issues
• For power on tasks hung at 95%, see Powering on a virtual machine pauses at 95% while waiting for a question to be answered

Troubleshooting power on task hangs

1. Wait at least 10 minutes for the task to fail.
2. Look for any other relevant errors/warnings in the vCenter or ESXi host UI. If there are any, see Failed to power on virtual machine
3. Find out if another task is still in progress. see Collecting information about tasks in VMware ESXi/ESX
   • Important: Do not cancel snapshot consolidation tasks. Doing so has a high chance of corrupting the VM's virtual disks.
     • You can monitor the consolidation task progress using the commands in Collecting information about tasks in VMware ESXi/ESX
   • If VM is running a task is not snapshot consolidation, you can attempt to cancel it.
   • If it won't cancel, power off the VM. If possible, power it off through the guest OS interface in an SSH or RDP session. If not, power it off in the vCenter UI. If won't power off in the vCenter UI, try to do so in the ESXi UI.
   • IF the VM won't power off, kill the VM processes per Unable to Power off a Virtual Machine in an ESXi host
4. If no tasks are found then please look at vmware.log for the VM for more information. The log file is found in: vmfs/volumes/<virtual machine datastore path>/vmware.log. Look for: 
   • Errors for input/output. If you see these move VM to different host or datastore.
   • Errors for CPU. If found, see VM Power On failed. No host is compatible with the VM
   • Errors for file locking. If found, remove the locks per Investigating virtual machine file locks on ESXi hosts
   • Errors for being unable to open a file, or any other error messages that clarify the problem. Please see Failed to power on virtual machine
5. If you try to access the vmware.log and get I/O errors then:
   • Troubleshoot for datastore/array problems, or
   • Restore or rebuild VM to healthier datastore.
6. If you find no other issues, migrate VM to another host and boot it from there.
7. If VM won't migrate, kill VM processes per Unable to Power off a Virtual Machine in an ESXi host
8. If problem persists:
   1. Right-click the VM object in the vCenter or ESXi UI, select "Edit settings..."
   2. Create new VM that copies all of the VM's settings
   3. Attach the old VM's virtual disks to the new VM and see if it will boot.

Additional Information

Consolidating/Committing snapshots in ESXi
Estimate the time required to consolidate virtual machine snapshots
Failed to power on virtual machine

Impact/Risks:
Performing the steps in this article can cause the following:

• The affected VM to be Unavailable.
• The ESXi host will become unavailable when restarting ESXi management agents, any running tasks should be completed prior to running this.
• Rebooting or restarting vCenter services can temporarily halt tasks which depend on vCenter Server, for example backups and VM deployment.