Snapshot consolidation fails and VM goes into hung state.

Products

• VMware vSphere ESXi

Issue/Introduction

• Snapshot consolidation initiated from a backup software might get failed due to the timed out error.
  
  
• During consolidation task, if another snapshot deletion task is initiated for the same Virtual machine, then vCenter will report below error in Virtual machine > Monitor > Tasks.

Another task is already in progress.



 

• /var/run/log/vmkernel.log indicate that the 0x89 command failed, with many instances of timeouts and aborted operations. Subsequently, the driver aborting all pending commands.

In var/run/log/vmkernel.log 2025-06-26T16:59:26.907Z cpu26:2097178)qlnativefc: vmhba64(86:0.0): qlnativefcStatusEntry:2077:C0:T2:L1 - FCP command status: 0x5-0x0 (0x8) portid=200300 oxid=0x85f cdb=890000 len=1024 rspInfo=0x0 resid=0x0 fwResid=0x0 host status = 0x8 device status = $

2025-06-26T16:59:26.907Z cpu30:2098260)NMP: nmp_ThrottleLogForDevice:3867: Cmd 0x89 (0x##########, 2097233) to dev "naa.########################" on path "vmhba64:C0:T2:L1" Failed:

2025-06-26T16:59:26.907Z cpu30:2098260)NMP: nmp_ThrottleLogForDevice:3875: H:0x8 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x########### CmdSN 0x281a265

• Virtual machine might go into a hung state.
  
  
  

Environment

VMware vSphere ESXi 7.x

VMware vSphere ESXi 8.x

Cause

Virtual machine get hung and operations like snapshot consolidation may fail due to IO failures leading to aborts and timeouts.

 

Sense code "H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x4b 0xc4" indicates target completing the SCSI command with aborted status. The ASC and ASCQ look proprietary to the storage target, and are indicative of some sort of Data Phase Error. This needs target vendor's input on what causes this error.

 

In var/run/log/vmkernel.log  2025-06-26T17:09:08.599Z cpu4:2098257)ScsiDeviceIO: 4115: Cmd(0x45b95fe8e8c8) 0x2a, CmdSN 0x42 from world 2104233 to dev "naa.###############################" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x4b 0xc4

2025-06-26T17:09:14.913Z cpu66:90089520)qlnativefc: vmhba1(13:0.0): qlnativefcTaskMgmt:2286:Task Mgmt abort on serial num 281a57e

2025-06-26T17:09:14.914Z cpu66:90089520)qlnativefc: vmhba1(13:0.0): qlnativefcEhAbort:2775:C0:T0:L1: Abort command succeeded -- 0

2025-06-26T17:09:14.914Z cpu66:90089520)qlnativefc: vmhba64(86:0.0): qlnativefcTaskMgmt:2286:Task Mgmt abort on serial num 281a57e

2025-06-26T17:09:14.914Z cpu66:90089520)qlnativefc: vmhba64(86:0.0): qlnativefcEhAbort:2728:SCSI command timeout counter incremented to 6296

 

D:0x2 => CHECK CONDITION,
Sense Key: 0xB => ABORTED COMMAND - Indicates that the device server aborted the command.
ASC/ASCQ: 0x4B/0x0 => DATA PHASE ERROR

 

Resolution

Engage storage team/vendor to further investigate the issue.