Receiving a SCSI Medium Error When Performing I/O to a Specific LBA on a LUN

Products

• VMware vSphere ESXi

Issue/Introduction

• When performing I/O operations on a LUN, the ESXi host reports SCSI Medium Errors in /var/log/vmkernel.log. These errors occur when attempting to read data from a specific Logical Block Address (LBA) on the storage device.
  Example log entries:
  • /var/run/log/vmkernel.log

YYYY-MM-DDTHH:MM:SSZ cpu4:2098170)ScsiDeviceIO: 4176: Cmd(0x45ba4164fc48) 0x28, CmdSN 0x5 from world 2103375 to dev "naa.600508b1001c####################" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x3 0x11 0x0 Medium Error, LBA: 15475715
YYYY-MM-DDTHH:MM:SSZ cpu1:2097804)ScsiDeviceIO: 4167: Cmd(0x45b8c12f8348) 0x28, CmdSN 0x334 from world 2099409 to dev "naa.614187705578####################" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x3 0x0 0x0 Medium Error, LBA: 31915520

• The error triggers also  during the transfer of VM snapshots to the backup storage. 

  • /var/run/log/hostd.log 
YYYY-MM-DDTHH:MM:Sec.MsecZ warning hostd[#####] [Originator@6876 sub=Libs opID=nbdmode-00000062########] [NFC ERROR]NfcAioGetMessage: Srv invalid msg hdr magic # 4, was expecting -15937#####
YYYY-MM-DDTHH:MM:Sec.MsecZ warning hostd[2100761] [Originator@6876 sub=Libs opID=nbdmode-00000062########] [NFC ERROR]NfcAioLogFatalSessionErrorLocked: A fatal session error occurred. The error was: 'NFC_SESSION_ERROR' (8)
YYYY-MM-DDTHH:MM:Sec.MsecZ warning hostd[#####] [Originator@6876 sub=Libs opID=nbdmode-00000062########] [NFC ERROR]NfcAioGetAndProcessMsg: Failed to receive an AIO message: NFC_SESSION_ERROR
YYYY-MM-DDTHH:MM:Sec.MsecZ info hostd[#####] [Originator@6876 sub=Libs opID=nbdmode-00000062########] [NFC INFO]NfcAioCloseFile: Closing file '[VRTX_Shared] VM_name/VM_name_1.vmdk'
YYYY-MM-DDTHH:MM:Sec.MsecZ error hostd[######] [Originator@6876 sub=Nfcsvc opID=nbdmode-00000062########] Read error from the nfcLib: NFC_SESSION_ERROR (done = yep)

Cause

SCSI Sense Data : 0x3 0x11 0x0

• Sense Key (0x3): MEDIUM ERROR
• Additional Sense Code (ASC/ASCQ):
• 0x11 / 0x00 = Unrecovered Read Error

Impact

• The storage device was unable to successfully read data from the specified Logical Block Address (LBA)

This indicates that the block is unreadable and the error could not be corrected by the device. The issue is not caused by ESXi, as the host successfully issued the command and received an error response from the device.

Resolution

Contact the storage array vendor for further assistance.