ESX vmkernel DellEMC Symmetrix/VMAX/PowerMAX "Baker 44" errors
Article ID: 372083
Updated On:
Products
Issue/Introduction
- Inability to issue I/Os
- Aborted commands
- Dropped FCP Frames
- CONDITION in the /var/log/vmkernel.log on ESX:
vmkernel: cpu4:2098228)NMP: nmp_ThrottleLogForDevice:3868: H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x44 0x0. Act:NONE. cmdId.initiator=0x430ca9a35b40 CmdSN 0xffff9189b6f23100
vmkernel: cpu4:2098228)ScsiDeviceIO: 4580: Cmd(0x45b952d46440) 0x2a, CmdSN 0xffff9189b6f23100 from world 2101240 to dev "naa.xxxxxxxxxxxxxxxxxxxxxxxxxxxx" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x44 0x0
Environment
ESX
Cause
The Sense Key (0xb) and Additional Sense Data/ASC Qualifier (0x44/0x0) translates to:
---------------------------
0b: ABORTED COMMAND
44/00: INTERNAL TARGET FAILURE
DellEMC uses this status code for multiple different conditions:
1. Inability to issue I/O to a track on the DellEMC array. This is commonly seen when there is a double fault on a RAID 5 volume and the LBAs are no accessible to the ESXi host.
2. The DellEMC array has aborted a command that was sitting in its queue for greater than 25 seconds. This is usually observed when the array is overloaded and it cannot process commands in a timely manner. This has typically been observed in environments when there is a flood of UNMAPs, which are CPU and IO intensive for the array, though there are other cases where FA/Director load was high enough to stall commands long enough on the frame to cross that 25 second timeout.
3. The DellEMC array will also use return this SCSI Sense information when it decides to dropped a FCP frame, which indicates poor fabric health or possibly a bad SFP.
Resolution
Contact DellEMC for collaborative support assistance with:
1. Inability to issue I/Os to the array.
2. Aborted commands from the array.
3. Dropped FCP frames (Fabric or SFP).
Additional Information
- Interpreting SCSI sense codes in VMware ESXi.
- For information on troubleshooting LUN connectivity, see Troubleshooting LUN connectivity issues on ESXi hosts (broadcom.com)
- For fibre channel LUNs: Troubleshooting fibre channel storage connectivity
Feedback
