VM power on task in vCenter is stuck

Products

• VMware vSphere ESXi
• VMware vSphere ESX 8.x

Issue/Introduction

• VM Power on task in vCenter appears to be stuck.
• You may also see VM Consolidation tasks or CLI commands taking a long time to complete or failing.
• The VMs are located on a local VMFS datastore.

 

Environment

• VMware ESXi 7.x
• VMware ESXi 8.x

Cause

• The local disks are disconnecting/reconnecting to the HBA that they are attached to or not responding to SCSI commands.
  
  
  • NOTE: Local disks disconnecting and reconnecting (flapping) on local HBA/RAID controllers often indicates issues with firmware compatibility, physical cabling, or power management.
    
    
    
• You can verify the issue from the affected ESXi host's /var/log/vmkernel.log by observing error messages similar to the following:
  
  
• • "Hotplug" events from the HBA driver
    
    2025-12-04T20:46:51.000Z In(182) vmkernel: cpu2:XXXXXXX)lsi_mr3_0000:10:00.0: megasas_hotplug_work: 580: event  code: 0x71
•  
  • High latency reported in the /var/log/vmkernel.log from the ESXi host:

2025-12-04T20:44:51.000Z In(180) vmkwarning: cpu22:XXXXXXX)WARNING: ScsiDeviceIO: 1780: Device naa.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX performance has deteriorated. I/O latency increased from average value of 15728 microseconds to 2859553 microseconds.

• • SCSI command aborts:
    
    2024-05-20T14:32:11.456Z cpu14:2097843)ScsiDeviceIO: 4321: Cmd(0x45b8c12f9140) 0x2a, CmdSN 0xabcd to dev "t10.NVMe____XXXXXXXXXXXXXXXXXXXXXXX" failed H:0x5 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
    
    
  • Additional hardware errors, such as:

2024-05-20T14:32:11.456Z cpu14:2097843)WARNING: NVMEMGMT : 1062 Controller 256 is not in operational state

Resolution

• Please verify the local HBA/RAID controller has supported drivers and firmware installed as per the Broadcom Compatibility Guide:
  
  Determining Network/Storage firmware and driver version in ESXi
  
  
• Ensure drive is properly connected/seated in the host.
• Please contact your hardware vendor to further troubleshooting of the disk(s), HBA, cabling, etc.