VM failing to power on with warning "The storage backing virtual disk has permanent device loss. You may be able to hot remove this virtual device from the virtual machine and continue after clicking Retry. Click Cancel to terminate this session."

Products

• VMware vCenter Server

Issue/Introduction

• The virtual machine unexpectedly powers off and cannot be restarted.
  
  
• "Heartbeat Failed" alert is triggered for the VM prior to the power-off event.
  
  
• Attempting to power on the VM fails with error message.
  "The storage backing virtual disk  has permanent device loss. You may be able to hot remove this virtual device from the virtual machine and continue after clicking Retry. Click Cancel to terminate this session"

Environment

VMware ESXi 7.x
VMware ESXi 8.x

Cause

The issue was caused by an underlying storage connectivity failure. The LUN hosting the VM’s disks became inaccessible, resulting in an unexpected power-off and preventing the VM from successfully restarting.

From ESXi var/run/log/vmkernel.log, error event with "sense code error H:0x1"

2025-12-11T20:43:25.449Z In(182) vmkernel: cpu28:12211274)ScsiDevice: 1808: Permanently inaccessible device :naa.################################ has no more open connections. It is now safe to unmount datastores (if any) and delete the device.
2025-12-11T20:43:25.926Z Wa(180) vmkwarning: cpu56:2097639)WARNING: ScsiPath: 5988: Path vmhba2:C0:T2:L95 is being removed.
2025-12-07T05:11:11.336Z In(182) vmkernel: cpu32:2097293)ScsiDeviceIO: 4686: Cmd(0x45da00c6bac0) 0x28, CmdSN 0x37b from world 12211277 to dev "naa.################################" failed H:0x1 D:0x0 P:0x0



Sense Code:
H:0x1- NO_CONNECT - The status is returned if the connection is lost to the LUN. This can occur if the LUN is no longer visible to the host from the array side or if the physical connection to the array has been removed.

Resolution

Recommend to fix the connectivity issue between Storage array and ESXi host.

Workaround:

To restore the VM, unregister and re-register the VM:

1. Unregister the VM:

   • Obtain the VMID: vim-cmd vmsvc/getallvms

   • Unregister the VM: vim-cmd vmsvc/unregister <VMID>

2. Re-register and Power On:

   • Re-register the VM (On a different host where there is no connectivity issue) - Add or Register a Virtual Machine (VM) in vCenter Server

   • Power-on the VM.

Additional Information