NVMe Device Unmounted or Under Permanent Error (nvme_statusScsiEnd: status:6281)

Products

• VMware vSAN

Issue/Introduction

Since it is an NVMe device, we do not see standard SCSI sense codes, e.g., 0x3 0x11 or 0x4 0x11, if there is a hardware failure since it does not use the SCSI protocol.
If there is a medium error, you will instead see the following message in vmkernel.log:

• nvme_statusScsiEnd: status:6281

Symptoms:
NVMe device is unmounted, marked as failed, or under permanent error.

Environment

VMware vSAN 6.x

Cause

The 81 in "nvme_statusScsiEnd: status:6281" indicates that there is an unrecoverable read error like a 0x3 0x11 on a SATA or SAS disk.

Resolution

The disk should be replaced as it is a physical hardware issue.

Additional Information

See the code 81h on page 83 in figure 131 in the following document: https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf