Error: "ESXi tracking filter not available" when consolidating snapshots
search cancel

Error: "ESXi tracking filter not available" when consolidating snapshots

book

Article ID: 317887

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

This articles provides a workaround to address snapshot consolidation issue due to failure in loading CBT module during ESXi boot process.


Symptoms:
  • Snapshot consolidation fails to complete post upgrade of ESXi version to 7.0 U1 using DELL Custom Image
  • Virtual Machine logs and Hostd logs will report failure in opening change tracking file (CTK) resulting with consolidation error "ESXi tracking filter not available"
  • Following errors in virtual machine log file </vmfs/volumes/<Datastore UUID>/<Virtual Machine Name>/vmware.log
2020-10-24T10:02:18.629Z| vcpu-0| A000: ConfigDB: Setting scsi0:1.redo = ""
2020-10-24T10:02:18.629Z| vcpu-0| I005: DISK: OPEN scsi0:1 '/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002.vmdk' persistent R[]
2020-10-24T10:02:18.631Z| vcpu-0| I005: AIOGNRC: Failed to open '/vmfs/devices/disks/5f46c24a-########-####-##########7c' : Could not find the file (60003) (0x2013).
2020-10-24T10:02:18.631Z| vcpu-0| I005: OBJLIB: Device '/vmfs/devices/disks/5f46c24a-########-####-##########7c' not present, retrying to create one.
2020-10-24T10:02:18.641Z| vcpu-0| I005: DISKLIB-LIB_BLOCKTRACK   : Resuming from change tracking info file /vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002-ctk.vmdk.
2020-10-24T10:02:18.645Z| vcpu-0| I005: DISKLIB-CTK   : Could not open change tracking file "/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test_1-000002-ctk.vmdk": Change tracking invalid
2020-10-24T10:02:18.651Z| vcpu-0| I005: DISKLIB-CTK   : Change tracking invalid; reinitializing.
2020-10-24T10:02:18.683Z| vcpu-0| I005: ConsolidateEnd: Snapshot consolidate complete: ESXi tracking filter not available (5).
  • /var/run/log/hostd.log
2020-10-26T15:48:44.261Z verbose hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] Removeallsnapshots received. Consolidate: true
2020-10-26T15:48:44.262Z info hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] State Transition (VM_STATE_ON -> VM_STATE_REMOVEALL_SNAPSHOT)
2020-10-26T15:48:44.359Z verbose hostd[3457085] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx opID=kg4v90cf-4586521-auto-2qaze-h5:70279734-6-78-1f35 user=vpxuser:company\exampleuser] Consolidate disks after snapshot removal.
2020-10-26T15:48:45.415Z verbose hostd[2102328] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/datastore1/5f46c24a-########-####-##########7c/Test.vmx] Consolidate Disks message: Consolidation failed for disk node 'scsi0:0': ESXi tracking filter not available.
  • Consolidation failed for disk node 'scsi0:1': ESXi tracking filter not available.

  • An error occurred while consolidating disks: ESXi tracking filter not available.



Environment

VMware vSphere 7.0.x

Cause

CBT Module fails to load during ESXi boot post upgrade is performed.

Note: Software FCoE is no longer supported in vSphere 7.0.

Resolution

This is a known issue affecting VMware ESXi 7.0 U1 and is fixed in ESXi 7.0 U2.

Workaround
Note: Consolidation of delta disk can be accomplished when a virtual machine is in a powered off state. 

Manually load CBT Module

  • Verify if the cbt module is enabled:
    esxcli system module list --enabled=true | grep cbt

  • Verify if the cbt module is loaded:
    esxcli system module list --loaded=true | grep cbt

  • To manually load the module:
    vmkload_mod cbt

The above changes do not persistent across boot. CBT Module load issues is also reported to be in relation to known boot bank issue. To address the issue, please follow the steps mentioned in KB Bootbank loads in /tmp/ after reboot of ESXi 7.0 Update 1 host.