Error: "ESXi tracking filter not available" when consolidating snapshots
Article ID: 317887
Updated On:
Products
VMware vSphere ESXi
Issue/Introduction
This article provides a workaround to address the snapshot consolidation issue caused by the failure to load the CBT module during the ESXi boot process.
Symptoms:
- Snapshot consolidation fails to complete after upgrading the ESXi version to 7.0 U1 using the DELL Custom Image.
- Virtual machine and Hostd logs will report a failure in opening the Change Tracking (CTK) file, resulting in the consolidation error: "ESXi tracking filter not available"
- The following errors can be found in the virtual machine log file:
</vmfs/volumes/<Datastore UUID>/<Virtual Machine Name>/vmware.log>
YYYY-MM-DDTHH:MM:SS.629Z| vcpu-0| A000: ConfigDB: Setting scsi0:1.redo = ""
YYYY-MM-DDTHH:MM:SS.629Z| vcpu-0| I005: DISK: OPEN scsi0:1 '/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002.vmdk' persistent R[]
YYYY-MM-DDTHH:MM:SS.631Z| vcpu-0| I005: AIOGNRC: Failed to open '/vmfs/devices/disks/5f46c24a-########-####-##########7c' : Could not find the file (60003) (0x2013).
YYYY-MM-DDTHH:MM:SS.631Z| vcpu-0| I005: OBJLIB: Device '/vmfs/devices/disks/5f46c24a-########-####-##########7c' not present, retrying to create one.
YYYY-MM-DDTHH:MM:SS.641Z| vcpu-0| I005: DISKLIB-LIB_BLOCKTRACK : Resuming from change tracking info file /vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002-ctk.vmdk.
YYYY-MM-DDTHH:MM:SS.645Z| vcpu-0| I005: DISKLIB-CTK : Could not open change tracking file "/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002-ctk.vmdk": Change tracking invalid
YYYY-MM-DDTHH:MM:SS.651Z| vcpu-0| I005: DISKLIB-CTK : Change tracking invalid; reinitializing.
YYYY-MM-DDTHH:MM:SS.683Z| vcpu-0| I005: ConsolidateEnd: Snapshot consolidate complete: ESXi tracking filter not available (5).
YYYY-MM-DDTHH:MM:SS.629Z| vcpu-0| I005: DISK: OPEN scsi0:1 '/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002.vmdk' persistent R[]
YYYY-MM-DDTHH:MM:SS.631Z| vcpu-0| I005: AIOGNRC: Failed to open '/vmfs/devices/disks/5f46c24a-########-####-##########7c' : Could not find the file (60003) (0x2013).
YYYY-MM-DDTHH:MM:SS.631Z| vcpu-0| I005: OBJLIB: Device '/vmfs/devices/disks/5f46c24a-########-####-##########7c' not present, retrying to create one.
YYYY-MM-DDTHH:MM:SS.641Z| vcpu-0| I005: DISKLIB-LIB_BLOCKTRACK : Resuming from change tracking info file /vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002-ctk.vmdk.
YYYY-MM-DDTHH:MM:SS.645Z| vcpu-0| I005: DISKLIB-CTK : Could not open change tracking file "/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002-ctk.vmdk": Change tracking invalid
YYYY-MM-DDTHH:MM:SS.651Z| vcpu-0| I005: DISKLIB-CTK : Change tracking invalid; reinitializing.
YYYY-MM-DDTHH:MM:SS.683Z| vcpu-0| I005: ConsolidateEnd: Snapshot consolidate complete: ESXi tracking filter not available (5).
- /var/run/log/hostd.log
YYYY-MM-DDTHH:MM:SS.261Z verbose hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] Removeallsnapshots received. Consolidate: true
YYYY-MM-DDTHH:MM:SS.262Z info hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] State Transition (VM_STATE_ON -> VM_STATE_REMOVEALL_SNAPSHOT)
YYYY-MM-DDTHH:MM:SS.359Z verbose hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] Consolidate disks after snapshot removal.
YYYY-MM-DDTHH:MM:SS.415Z verbose hostd[2102328] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx] Consolidate Disks message: Consolidation failed for disk node 'scsi0:0': ESXi tracking filter not available.
YYYY-MM-DDTHH:MM:SS.262Z info hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] State Transition (VM_STATE_ON -> VM_STATE_REMOVEALL_SNAPSHOT)
YYYY-MM-DDTHH:MM:SS.359Z verbose hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] Consolidate disks after snapshot removal.
YYYY-MM-DDTHH:MM:SS.415Z verbose hostd[2102328] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx] Consolidate Disks message: Consolidation failed for disk node 'scsi0:0': ESXi tracking filter not available.
- Consolidation failed for disk node 'scsi0:1': ESXi tracking filter unavailable.
- An error occurred during disk consolidation: "ESXi tracking filter was not available.
Environment
VMware vSphere 7.0.x
Cause
The CBT module fails to load during the ESXi boot process after an upgrade.
Note: Software FCoE (Fibre Channel over Ethernet) is no longer supported in vSphere 7.0.
Resolution
This is a known issue affecting VMware ESXi 7.0 U1 and is fixed in ESXi 7.0 U2.
Workaround
Note: Consolidation of delta disk can be accomplished when a virtual machine is in a powered off state.
Manually load CBT Module
- Verify if the cbt module is enabled:
esxcli system module list --enabled=true | grep cbt - Verify if the cbt module is loaded:
esxcli system module list --loaded=true | grep cbt - To manually load the module:
vmkload_mod cbt
The above changes do not persistent across boot. CBT Module load issues is also reported to be in relation to known boot bank issue. To address the issue, please follow the steps mentioned in KB Bootbank loads in /tmp/ after reboot of ESXi 7.0 Update 1 host.
Feedback
Yes
No
Powered by
