ESXi Upgrade Failure due to Inaccessible Boot LUN

Products

• VMware vSphere ESXi

Issue/Introduction

This article details the troubleshooting and resolution steps for an ESXi host upgrade failure where the boot LUN becomes inaccessible or is reported as dead during the upgrade process. 

• The esxcli command returns a non-zero return code (1).
• Logs report: VmkCtl: skipping bad volume in boot filesystem search: Cannot open volume: and Unable to find device with name.
• Host hardware diagnostics indicate a dead or failing boot LUN. For Example: vCenter GUI reports error: 

"Lost connectivity to the device t10.NVMe_______________#########__________########## backing the boot filesystem

/vmfs/devices/disks/t10.NVMe___________#######_________#########. As a result, host configuration changes will not be saved to persistent storage."

 

Environment

• VMware vSphere ESXi 8.0.x

Cause

A hardware-level failure of the boot LUN occurs, rendering the boot volume inaccessible. The ESXi kernel attempts to locate the device for partition modification, but the volume is marked as dead by the storage layer, causing the upgrade process to abort.

Resolution

1. Validate Hardware Health:
   1. Access the server via iDRAC/IPMI or local console.
   2. Review hardware logs for controller or disk-level faults.
   3. Confirm the status of the boot device.
2. Hardware Remediation:
   1. If the device is reported as failed, engage the hardware vendor to validate and replace the physical drive or controller.
   2. If the device status is ambiguous, perform a cold reboot from the iDRAC to force a full re-initialization of the storage controller and host hardware.
3. Re-attempt Upgrade:
   1. Once hardware health is confirmed/restored, re-run the upgrade command.
   2. If the issue persists, contact Support for further log analysis.

 

Additional Information

To speak with a customer representative or a Support Engineer see Contact Support.