Windows guest OS boot failure with NVMe on VM Hardware version 20 and 21

Products

• VMware vSphere ESXi

Issue/Introduction

• Windows 2025 guest OS does not boot up while using NVMe controller for the disk and VM Hardware version is 20 or 21.
• This issue is not see when the VM is booted with NVMe controller and HW version 19.
• If the NVMe controller is changed to SCSI controller, the VM boots up normally.
• When powering ON the VM, /vmfs/volumes/<datastore_name>/<vm_name>/vmware.log shows below entries:
  
  <date><time> In(05) vcpu-0 - NVME-VMK: nvme0:0: FLUSH QUEUE Command failed. Status: 0x80/0x1.
<date><time> In(05) vcpu-0 - NVME-VMK: nvme0:0: FLUSH QUEUE Command failed. Status: 0x80/0x1.
<date><time> In(05) vcpu-0 - Guest: Status upon boot failure: No Media
<date><time> In(05) vcpu-0 - NVME-VMK: nvme0:0: FLUSH QUEUE Command failed. Status: 0x80/0x1.
<date><time> In(05) vcpu-0 - NVME-VMK: nvme0:0: FLUSH QUEUE Command failed. Status: 0x80/0x1.
<date><time> In(05) vcpu-0 - Msg_Post: Warning
<date><time> In(05) vcpu-0 - [msg.Backdoor.OsNotFound] No operating system was found. If you have an operating system installation disc, you can insert the disc into the system's CD-ROM drive and restart the virtual machine.


• /var/run/log/vmkernel.log on ESXi host during the same time shows below entries:
  
  <date><time> Wa(180) vmkwarning: cpu#:#######)WARNING: VNVMEScsi: 823: NVME Opcode: 0x0 Not Supported, handle ID: 0x#######0
<date><time> In(182) vmkernel: cpu#:#######)VNVME: 338: Error status: Not supported converted to: 0x80:0x1
<date><time> Wa(180) vmkwarning: cpu#:#######)WARNING: VNVMEScsi: 823: NVME Opcode: 0x0 Not Supported, handle ID: 0x#######0
<date><time> In(182) vmkernel: cpu#:#######)VNVME: 338: Error status: Not supported converted to: 0x80:0x1

Environment

VMware vSphere ESXi 8.x

Resolution

This issue is resolved in VMware ESXI 8.0u3i and later, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

Workarounds:

1. Power off the VM and change the NVMe controller to SCSI controller.
2. Create a new VM with VM Hardware version as 19.