Auto UNMAPs stop working for VMFS/VVOL datastores created out of NVMe devices after upgrading ESXi to 8.x

Products

• VMware vSphere ESXi
• VMware vSphere ESXi 8.0

Issue/Introduction

Auto UNMAP commands are sent successfully from a 7.x ESXi host to the same set of NVMe VMFS datastores, but fails from a host upgraded to 8.x.

Issue is observed on VMFS or VVOL datastores created out of NVMe devices. VMFS datastores using iSCSI and FC protocols are not impacted.

Driver and Firmware for the NVMe adapter are as per Broadcom VCG.

To check UNMAP stats, SSH into ESXi host:
esxtop > option u >  option f > option o (uncheck other fields as necessary to fit UNMAP data into window) > press enter to find UNMAP stats under DELETE column

 

Environment

VMware vSphere ESXi 8.x

Cause

VMkernel.log from 8.x host:
2024-12-11T20:34:12.358Z In(182) vmkernel: cpu38:2099016 opID=436439f6)NvmeFds: 3105: Unable to fetch UNMAP Attributes for device eui.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2024-12-11T20:34:12.360Z Wa(180) vmkwarning: cpu20:2171422)WARNING: NVMEIO:2644 command 0x45dacd8946c0 failed: ctlr 273, queue 0, psaCmd 0x45dac00cb500, status 0xf, opc 0x6, cid 31, nsid 0
2024-12-11T20:34:12.360Z Wa(180) vmkwarning: cpu20:2171422)WARNING: NVMEPSA:217 Complete vmkNvmeCmd: 0x45dacd8946c0, vmkPsaCmd: 0x45dac00cb500, cmdId.initiator=0x453a58c9b408, CmdSN: 0x0, status: 0xf

grep -r "Unable to fetch UNMAP" * | wc -l
4200

As seen from log snippet: 
PSA is issuing identify command (opcode 0x6) to learn the UNMAP attributes from the target. However, identify command is failed by the target with status 0xf.

Resolution

VMware is aware of the issue, a fix will be provided in a future release.