Explaining DellEMC Symmetrix/VMAX/PowerMAX "Baker 44" errors

Products

• VMware vSphere ESXi
• VMware vSphere ESXi 5.0
• VMware vSphere ESXi 5.x - View
• VMware vSphere ESXi 6.0
• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 8.0
• VMware vSphere ESXi 5.5

Issue/Introduction

CONDITION in the vmkernel.log on ESXi:

vmkernel: cpu4:2098228)NMP: nmp_ThrottleLogForDevice:3868: H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x44 0x0. Act:NONE. cmdId.initiator=0x430ca9a35b40 CmdSN 0xffff9189b6f23100
vmkernel: cpu4:2098228)ScsiDeviceIO: 4580: Cmd(0x45b952d46440) 0x2a, CmdSN 0xffff9189b6f23100 from world 2101240 to dev "naa.xxxxxxxxxxxxxxxxxxxxxxxxxxxx" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x44 0x0

Resolution

The Sense Key (0xb) and Additional Sense Data/ASC Qualifier (0x44/0x0) translates to:
---------------------------
0b: ABORTED COMMAND
44/00: INTERNAL TARGET FAILURE

DellEMC uses this status code for multiple different conditions:

1. Inability to issue I/O to a track on the DellEMC array. This is commonly seen when there is a double fault on a RAID 5 volume and the LBAs are no accessible to the ESXi host.

2. The DellEMC array has aborted a command that was sitting in its queue for greater than 25 seconds. This is usually observed when the array is overloaded and it cannot process commands in a timely manner. This has typically been observed in environments when there is a flood of UNMAPs, which are CPU and IO intensive for the array, though there are other cases where FA/Director load was high enough to stall commands long enough on the frame to cross that 25 second timeout.

3. The DellEMC array will also use return this SCSI Sense information when it decides to dropped a FCP frame, which indicates poor fabric health or possibly a bad SFP.