VM is stuck during Veeam backup

Products

• VMware vSphere ESXi

Issue/Introduction

• VM is stuck during Veeam backup
• VM vmware log stops during the Snapshot without error
• The createSnapshot task is timedout from vCenter vpxd log:

error vpxd[06121] [Originator@6876 sub=Default opID=393ddb6] [VpxLRO] -- ERROR task-1224485 -- 522fb330-######0d5d(52c0a4b2-f2bc-48a5-b741-656fb10feccd) -- vm-### -- vim.VirtualMachine.createSnapshot: :vim.fault.Timedout
--> Result:
--> (vim.fault.Timedout) {
-->    faultCause = (vmodl.MethodFault) null, 
-->    faultMessage = <unset>
-->    msg = ""
--> }
--> Args:
--> 
--> Arg name:
--> "VEEAM BACKUP TEMPORARY SNAPSHOT"
--> Arg description:
--> "Please do not delete this snapshot. It is being used by Veeam Backup."
--> Arg memory:
--> false
--> Arg quiesce:
--> false

• ESXi hostd log keeps showing:

Wa(164) Hostd[2100035]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/####/#####.vmx opID=WorkQueue-73ac4477-c7-3195 sid=52ea2007 user=vpxuser:<no user>] File - failed to get objectId, '/vmfs/volumes/####/####/####.vmx': Operation not supported (11)
In(166) Hostd[2100035]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/####/####/####.vmx opID=WorkQueue-73ac4477-c7-3195 sid=52ea2007 user=vpxuser:<no user>] Error retrieving file layout : N3Vim5Fault8Timedout9ExceptionE(Fault cause: vim.fault.Timedout
In(166) Hostd[2100035]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/####/####/####.vmx opID=WorkQueue-73ac4477-c7-3195 sid=52ea2007 user=vpxuser:<no user>] Skip caching of external disk paths due to incomplete layout.

• The VM is protected by Veeam CDP:

####.vmdk:ddb.iofilters = "veecdp"
####vmdk:ddb.sidecars = "veecdp_1,####33ed2.vmfd"

• The ESXi host upgraded recently

Environment

VMware vSphere ESXi

Cause

The VM is protected by Veeam CDP from VMDK while the CDP is disabled on Veeam

Resolution

Contact Veeam to delete the CDP and check if any issue on iofilters.

Recommend to schedule maintenance window as VM may be stuck during the CDP deleting the same as Veeam backup