Snapshot failed with Error - "Error occurred while taking a snapshot: Operation failed"
search cancel

Snapshot failed with Error - "Error occurred while taking a snapshot: Operation failed"

book

Article ID: 388337

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:

  • Unable to create manual snapshot on the Virtual Machine.
     
On the vSphere Client, navigate to Virtual Machine in the inventory > Click the Monitor tab> Under Tasks and Events, select Tasks.

    Environment

    VMware vSphere 7.x
    VMware vSphere 8.x

    Cause

    Issue is due the third-party jetdr I/O filter module which is blocking and failing the snapshot task during the prepare phase. Snapshot can fail for a multi-disk VM if the backup appliance does not have enough resources (e.g., attempting to back up too many VMs/disks concurrently). As result, some disks of the affected VM may not be snapshotted. In such a case, JetStream DR restores/returns to a pre-snapshot state for already snapshotted disks. 

    Virtual machine Logs: /vmfs/volumes/[datastore-name]/[vm-name]/vmware.log below errors indicates the issue with the third-party jetdr module

    2025-02-06T03:23:49.631Z Er(02):vmx - DISKLIB-LIB_CREATE :DiskLib_NotifySnapshotPrepare: FiltLib failed to noilify snapshot prepare: Operation completes asynchronously
    2025-02-06T03:23:49.651Z In(05):vmx - DISKLIB-CBT: ChangeTrackerESX_CreateMirror: Created mirror node /vmfs/devices/<vm/****-cbtmirror.
    2025-02-06T03:23:49.666Z In(05):vmx - SNAPSHOT: SnapshotPrepareTakeDoneCB: Failed to prepare filter.
    2025-02-06T03:23:49.752Z In(05):vmx - SNAPSHOT: SnapshotPrepareTakeDoneCB: Prepare phase complete (Operation failed,
    2025-02-06T03:23:49.752Z In(05):vmx - SnapshotVMXPrepareTakeDoneCB: Prepare phase failed: Operation failed (5).
     
    025-02-06T03:23:49.560Z In(05) vmx - DISKLIB-CBT :ChangeTrackerESX_CreateMirror: Created mirror node /vmfs/devices/svm/3f2c8886-371b1c3b-cbtmirror.
    2025-02-06T03:23:49.570Z In(05) vmx -I>FLTR_MAIN: FioCbkSnapshot(1824): VMIOF_SNAPSHOT_PREPARE entered for VM.vmdk, opMode=2.
    2025-02-06T03:23:49.570Z Wa(03) vmx -I>FLTR_MAIN: FioCbkSnapshot(1945): Unable to take snapshot of VM_1.vmdk as protection start is not completed yet.
    025-02-06T03:23:49.570Z In(05) vmx - FiltLib: jetdr: Notification completed with error: "Operation failed" (1).
    2025-02-06T03:23:49.570Z In(05) vmx -FiltLib: jetdr: filter failed notification processing: "Operation failed" (1).

    Resolution

    Workaround:

    Snapshot may fail due to insufficient resource, so wait for a while before attempting to take the snapshot again and then retake the snapshot.

    If the above operation fails It is recommended to involve the third-party (Jetstream) vendor to further investigate the issue.

    Additional Information

    I/O filters are software components that can be installed on ESXi hosts and can offer additional data services to virtual machines. The filters process I/O requests, which move between the guest operating system of a virtual machine and virtual disks. The I/O filters can be offered by VMware or created by third parties through vSphere APIs for I/O Filtering (VAIO).

    About JetStream ESXi host components (IO Filter packages):

    JetStream software installed on each ESXi host configured for JetStream DR. The host driver intercepts the vSphere VMs I/O and sends the replication data to the DRVA. The IO filters also monitor relevant events, such as vMotion, Storage vMotion, snapshots, etc.