Module Disk power on failed - I/O error accessing change tracking file File system specific implementation

Products

• VMware vSphere ESXi

Issue/Introduction

You cannot power ON a virtual machine. 

Task Name
Power On virtual machine
Status
File system specific implementation of GetPageRef[file] failed Failed to start the virtual machine. Module Disk power on failed. Cannot open the disk '/vmfs/volumes/5008####-ea75####/ESXiVM/ESXiVM.vmdk' or one of the snapshot disks it depends on. I/O error accessing change tracking file File system specific implementation of Ioctl[file] failed File system specific implementation of Ioctl[file] failed
Initiator

Target: ESXiVM
Server: ESXi-host.datacenter.com
Error stack:
Module Disk power on failed.
Cannot open the disk '/vmfs/volumes/5008####-ea75####/ESXiVM/ESXiVM.vmdk' or one of the snapshot disks it depends on.
I/O error accessing change tracking file
File system specific implementation of Ioctl[file] failed
File system specific implementation of Ioctl[file] failed
File system specific implementation of GetPageRef[file] failed

Environment

VMware ESXi 7.0.x

Cause

The VM is unable to read the CTK file because its 0 bytes. 

Resolution

[root@ESXi-host:/vmfs/volumes/5008####-ea75####/ESXiVM] ls -lah
 
-rw-rw-rw-    1 root     root           0 Aug 23 02:13 ESXiVM-ctk.vmdk
-rw-------    1 root     root        1.0T Aug 23 01:18 ESXiVM-flat.vmdk
-rw-------    1 root     root         625 Aug 22 14:11 ESXiVM.vmdk

Create a new directory inside the VM directory and move the 'ESXiVM-ctk.vmdk' file to it. Now you should be able to power ON the VM. 

Additional Information

Consolidation of VM may fail with error 'I/O error accessing change tracking file'

• The following error appears during consolidation: “An error occurred while consolidating disks: I/O error accessing change tracking file.”
• The virtual machine in a powered off state.

The vmware.log for the VM contains events similar to the following:
YYYY-MM-DDTHH:MM:SS.020Z In(05) vcpu-0 - Turning off snapshot info cache.
YYYY-MM-DDTHH:MM:SS.020Z In (05) vcpu-0 - Turning off snapshot disk cache.
YYYY-MM-DDTHH:MM:SS.020Z In(05) vcpu-0 - ConsolidateEnd: Snapshot consolidate complete: I/0 error accessing change tracking file .

To complete the consolidation process and power on the VM, disable CTK, move the CTK files from the VM folder to a temporary directory, perform the consolidation, then re-enable CTK and power on the VM. 

• Disable ctk  :

1. 1. Power off the virtual machine.
   2. Right-click the virtual machine and click Edit Settings.
   3. Click the  Advanced Parameters tab..
   4. Set the ctkEnabled parameter to FALSE for the ctkEnabled setting for the VM 

Note: before disabling ctk , ensure that there are no snapshots on the virtual machine

• Move all the old ctk files from VM folder to a backup folder example - ctk_backup
•  Create a dedicated backup folder

 # mkdir ctk_backup

 

#  mv *-ctk.vmdk ctk_backup/

• Retry the consolidation
• After the consolidation completes successfully
• Re-enable the ctk :

1. 1. Open the vSphere Client and select a powered-off virtual machine.
   2. Go to  Edit settings of VM > Advanced Parameters> Advanced Configuration Parameters  (Search for ctk in Attribute filter)
   3. Change The option ctkEnabled = "TRUE"

Note: Ensure that there are no snapshots on the virtual machine before enabling Changed Block Tracking

• Power on VM 

For more details refer to Changed Block Tracking (CBT) on virtual machines