Finding consolidation tasks that are at 99%
Monitor using watch in ESXi 6.x/ 7.x
For 5.x, see Monitor snapshot deletion using esxtop command (2146232)
Monitor using vim-cmd in ESXi
Finding consolidation tasks that are at 99%
- Log in as root to the ESXi host using SSH.
- Navigate to the virtual machine directory containing vmdk virtual disk files. Run the cd command to change the current directory.
For example:
cd /vmfs/volumes/Datastore_name/Virtual_Machine_name/
- Run less testdev02.vmx | grep vmdk to list vmdk (disks) of the VM in the vmx (configuration file of VM):
For example:
less testdev02.vmx |grep vmdk
scsi0:0.fileName = "testdev02.vmdk"
scsi0:1.fileName = "testdev02_1-000003.vmdk"
scsi0:2.fileName = "testdev02_2-000002.vmdk"
- Run this command to list the snapshot files in the directory:
- less testdev02.vmx | grep *sesparse.vmdk
- For VMFS5 volume: less testdev02.vmx | grep *delta.vmdk
Listing the snapshot of the VM.
-rw------- 1 root root 18.2G Oct 29 10:04 testdev02_2-000001-sesparse.vmdk
-rw------- 1 root root 943.6G Oct 29 10:09 testdev02_1-000002-sesparse.vmdk
-rw------- 1 root root 1.8T Oct 29 10:09 testdev02_1-000001-sesparse.vmdk
-rw------- 1 root root 1.7G Oct 29 20:03 testdev02_2-000002-sesparse.vmdk
-rw------- 1 root root 3.1T Oct 29 20:03 testdev02_1-000003-sesparse.vmdk
Note: Sesparse is a default format for all delta disks on the VMFS6 datastores. On VMFS5, Sesparse is used for virtual disks of the size 2 TB and larger.
- In this example we have snapshots approximately 6 TB and this could be the main reason for the progress of the task. The process will continue to show 99% until the snapshot removal task completes and VM is using the flat files of the VMDK.
- Find the consolidation task with this command: vim-cmd vimsvc/task_list
For example:
[root@ESXi:~] vim-cmd vimsvc/task_list
(ManagedObjectReference) [
'vim.Task:haTask-91-vim.vm.Snapshot.remove-321615207',
'vim.Task:haTask-ha-host-vim.host.ImageConfigManager.installDate-321689752',
'vim.Task:haTask-ha-host-vim.host.ServiceSystem.start-321689774',
'vim.Task:haTask-ha-host-vim.host.ServiceSystem.refresh-321689779'
]
- View the details of the consolidation with this command: vim-cmd vimsvc/task_info <task_name_from_step_6>
[root@ESXi:~] vim-cmd vimsvc/task_info haTask-91-vim.vm.Snapshot.remove-321615207
(vim.TaskInfo) {
key = "haTask-91-vim.vm.Snapshot.remove-321615207",
task = 'vim.Task:haTask-91-vim.vm.Snapshot.remove-321615207',
description = (vmodl.LocalizableMessage) null,
name = "vim.vm.Snapshot.remove",
descriptionId = "vm.Snapshot.remove",
entity = 'vim.VirtualMachine:81',
entityName = "TESTDEV",
locked = <unset>,
state = "running",
cancelled = false,
cancelable = true,
error = (vmodl.MethodFault) null,
result = <unset>,
progress = 99,
reason = (vim.TaskReasonUser) {
userName = "vpxuser"
},
queueTime = "2019-10-29T10:08:21.403596Z",
startTime = "2019-10-29T10:08:21.404185Z",
completeTime = <unset>,
eventChainId = -1078812089,
changeTag = <unset>,
parentTaskKey = <unset>,
rootTaskKey = <unset>,
activationId = <unset>
}
Monitor using watch in ESXi 6.x/7.x
To monitor the VMDK snapshot and base disks which are currently being updated, run this watch command:
This command refreshes the file referred by the on going task every 2 seconds.
# watch -d 'ls -luth | grep -E "delta|flat|sesparse"'
where:
-d highlights the differences between successive updates
In addition, run this command to monitor the time stamp update of the base disks to confirm if the process is working. The result lists the files by their modification date.
ls -lrth |grep -E "flat|delta|sesparse"
Notes:
- In ESX/ESXi 4.0 Update 2 and later, the process works differently in that the data in snapshots (deltas) are written directly to the base disk (flat).
- If there are more than 10 snapshots, run this command to monitor the snapshot commit process and to prevent the screen from filling with many files:
#while true;do date;ls -lht *vmdk|head -10;echo ________;sleep 3;done
- You can quit the consolidation process monitoring by pressing Ctrl + C.