Snapshot consolidation failed for VM on vVOLs storage
Article ID: 411756
Updated On:
Products
Issue/Introduction
A snapshot consolidation will fail for a VM housed on vVOls storage resulting in an orphaned snapshot file.
From the vmware.log file for the target VM you may see an error similar to the following:
####-##-##t##:##:##.###Z In(05) vcpu-0 - DISKLIB-VMFS : "vvol://################-################/rfc4122.0504098b-86c5-4c30-a861-44fa310e18c4" : open successful (1114113) size = 0, hd = 0. Type 3####-##-##t##:##:##.###Z In(05) vcpu-0 - DISKLIB-LIB_BLOCKTRACK : Resuming from change tracking info file /vmfs/volumes/vvol:################-################/rfc4122.cb9d4c61-b741-4377-a182-39b0c96fb033/VMNAME-ctk.vmdk.####-##-##t##:##:##.###Z In(05) vcpu-0 - DISKLIB-CTK : Unlinking change tracking info /vmfs/volumes/vvol:################-################/rfc4122.cb9d4c61-b741-4377-a182-39b0c96fb033/VMNAME-ctk.vmdk.####-##-##t##:##:##.###Z In(05) vcpu-0 - DISKLIB-CTK : Could not unlink tracking file "/vmfs/volumes/vvol:################-################/rfc4122.cb9d4c61-b741-4377-a182-39b0c96fb033/VMNAME-ctk.vmdk".####-##-##t##:##:##.###Z In(05) vcpu-0 - DISKLIB-LIB_BLOCKTRACK : Warning: could not stop change tracking info: Could not delete change tracking file (10).####-##-##t##:##:##.###Z In(05) vcpu-0 - DISKLIB-VMFS : "vvol://################-################/rfc4122.0504098b-86c5-4c30-a861-44fa310e18c4" : closed.####-##-##t##:##:##.###Z Er(02) vcpu-0 - DISKLIB-LIB : DiskLibUnlinkInt: Disk delete failed to complete { result:2620, errMsg:'Could not delete change tracking file', fileName:'/vmfs/volumes/vvol:################-################/rfc4122.cb9d4c61-b741-4377-a182-39b0c96fb033/VMNAME.vmdk'}####-##-##t##:##:##.###Z In(05) vcpu-0 - SNAPSHOT: SnapshotFixDisksAfterCombineNative /vmfs/volumes/vvol:################-################/rfc4122.cb9d4c61-b741-4377-a182-39b0c96fb033/VMNAME.vmdk unlink fails with Could not delete change tracking file (2620)
From the vmkernel.log on the ESX host you may see an error similar to the following:
####-##-##t##:##:##.###Z In(182) vmkernel: cpu2:2099415 opID=65f0f56)VVol: VVolAllocAndInitDevice:9225: Creating a device for rfc4122.0504098b-86c5-4c30-a861-44fa310e18c4 (Type VVOL_OBJTYPE_VMDK)####-##-##t##:##:##.###Z In(182) vmkernel: cpu24:6105324)NMP: nmp_ThrottleLogForDevice:3893: Cmd 0x89 (0x45b97d24c840, 2097208) to dev "eui.################################" on path "vmhba64:C0:T0:L3" Failed:####-##-##t##:##:##.###Z In(182) vmkernel: cpu24:6105324)NMP: nmp_ThrottleLogForDevice:3898: H:0x5 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x4307d03e2b40 CmdSN 0x88484 sllid: e200000008320000####-##-##t##:##:##.###Z Wa(180) vmkwarning: cpu24:6105324)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:235: NMP device "eui.################################" state in doubt; requested fast path state update...####-##-##t##:##:##.###Z In(182) vmkernel: cpu24:6105324)ScsiDeviceIO: 4670: Cmd(0x45b97d24c840) 0x89, cmdId.initiator=0x4307d03e2b40 CmdSN 0x88484 from world 2097208 to dev "eui.################################" failed H:0x5 D:0x0 P:0x0 Cancelled from driver layer.####-##-##t##:##:##.###Z In(182) vmkernel: cpu24:6105324)Cmd count Active:63 Queued:0 sllid: e200000008320000####-##-##t##:##:##.###Z In(182) vmkernel: cpu24:6105324)VVol: 2770: [vvolDev: rfc4122.82267d63-b4ee-4a1b-9ae8-b9b9cb2a37e2] WRITE command (ioFlags 0x10, cmd 39 (FDS ATS-block ioctl)) failed: Timeout (0xbad0021)
Environment
ESX 8.x
Cause
The cause of this issue is still under investigation. If you encounter this error, it is suggested to open an case with GS for further review.
Resolution
There is no known resolution at this time.
Feedback
