Bad block error occurs frequently on Windows guest OS

Products

• VMware vSphere ESXi

Issue/Introduction

In ESX 7.0U2 release, for NVME adapter if guest issues UNMAP/DSM-TRIM command on a thick disk, accidentally unmap gets enabled at VSCSI layer for this disk and hence guest continues to issue unmap commands.

These commands pass through to backing storage layer mainly VMFS, which are rejected with NOT_SUPPORTED check condition. But now this check condition gets reported as generic IO_ERROR to guest. This results in Bad block error on windows OS.

vmware.log also prints following error:

NVME-VMK: nvme0:0: DATASET MANAGEMENT Command failed. Status: 0x0/0x4.

Linux also prints similar error in kernel.log.

Symptoms:
Issue happens only when thick provisioned disks are attached to NVME adapter on 7.0U2.

Environment

VMware vSphere ESXi 7.0.0

Cause

This is not actually bad block or data corruption error. Unmap issued by guest on thick disk failed with wrong error code.

Resolution

This is a known issue affecting ESXi 7.0U2.

Workaround:
This is not actually bad block or data corruption error. Unmap issued by guest on thick disk failed with wrong error code.


1. Switch to PVSCSI controller. This issue doesn't happen with disks attached to PVSCSI adapter.

2. If NVME adapter is necessary, then only workaround is to create VM snapshot, which makes thick disks appear thin to guest (This is because snapshot disks supports unmap for VMFS based disks) . But this can slow down performance.