Understanding virtual machine snapshots within Virtual Volumes (vVols)

Products

VMware vSphere ESXi

Issue/Introduction

The purpose of this article is to explain how virtual machine snapshots behave in virtual machines that are stored on a VMware Virtual Volume (vVols).

vVols constitutes five major components namely vVols Device, Protocol End Point, Storage Container, VASA Provider and Array and all these components are managed/used/handled by different components in vSphere Stack such as Virtual Center (VASA, SPBM), ESXi (Hostd, VVOLD, VVOL FDS Driver). It becomes necessary to get a holistic view of environment and configuration.

Characteristics of vVols:

No File System.
ESXi manages the array through VASA (vSphere APIs for Storage Awareness) APIs.
Arrays are logically partitioned into containers, called Storage Containers.
Virtual machine disks, called vVols, stored natively on the Storage Containers.
IO from ESXi host to the storage array is addressed through an access point called, Protocol Endpoint (PE).
Data Services are offloaded to the array. Snapshot, Replication, Encryption.
Managed through storage policy-based management (SPBM) framework.

snapshot vVols

The normal snapshot mechanism for virtual machines which reside on VMFS volumes, relies on a redo log format snapshot. For more information about this type of snapshot, see:

Snapshot on VMFS Datastore	Snapshot on vVols
When a snapshot of a virtual machine residing on a VMFS datastore is taken a child delta disk is created, the parent disk is the base virtual disk (VMDK).	When virtual machines are stored on a Virtual Volume (vVols), the first thing to remember is that with a virtual machine on snapshot is always running from the base VMDK disk.
The parent disk is considered a point-in-time (PIT) copy. When the snapshot is created the virtual machine now runs from the snapshot virtual disk (delta VMDK).	When a snapshot is taken, virtual machine no longer has its running point on a snapshot delta as it is pointed to base disk as mentioned above.
Base disk remains Read only as the Reads and Writes happen on the snapshot disk (delta disk).	Base disk remains RW for snapshot on vVols.
All new writes to the virtual machine's disks are recorded in the delta VMDK, disk read operations are served by the base VMDK and any data written since the snapshot creation is read from the delta VMDK disk(s).	The delta VMDK is responsible for maintaining a point-in-time (PIT) copy of the data which means that when the virtual machine sends I/O to the base VMDK disk, the delta VMDK is responsible for tracking the original data block.
Snapshots can negatively affect the performance of a virtual machine. Performance degradation is based on how long the snapshot or snapshot tree is in place, the depth of the tree, and how much the virtual machine and its guest operating system have changed from the time you took the snapshot. Amount of time VM takes to power on will depend on the hierarchy of snapshot chain along with the size of Delta/sesparse.	With VVols, when you choose take snapshot of VM, VMware does not have any performance-impact while creating delta VMDK files that were traditionally used, but instead VMware entirely offloads this process to the array.
If a virtual machine has virtual hard disks larger than 2 TB, snapshot operations can take much longer to finish.	The snapshots are handled by the array and the copy-on-write (COW) operations that are needed to maintain the snapshot. As a result, the I/O from the VM to the disk does not have the performance penalty because the VM is running on an active snapshot.

This diagram illustrates a virtual machine residing on a VMFS datastore, and what the snapshot structure looks like when the first and second snapshots are created.

This diagram illustrates a virtual machine residing on a vVols datastore, and what the snapshot structure looks like when the first and second snapshots are created.

Environment

VMware vSphere ESXi 7.0.0
VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7

Resolution

A virtual machine disk descriptor file on a Virtual Volume (vVols) no longer points to a flat VMDK file, it references a vVols object location.

This is an example of a virtual machine disk (VMDK) descriptor file before a snapshot is taken:

[root@ESXi60] cat testVM.vmdk

# Disk DescriptorFile
version=4
encoding="UTF-8"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfs"

# Extent description
RW 4194304 VMFS "vvol://1b38adb34b7f45f8-9abcb8e9eadba042/rfc4122.0a0a1e74-d7c4-4217-a49e-06281db2cffa"

# The Disk Data Base
#DDB

ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "261"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "847f9654f70db75004bc90fbfffffffe"
ddb.uuid = "60 00 C2 91 c4 12 18 3b-5b 5c 70 bb 92 f6 9a e3"
ddb.virtualHWVersion = "11"

When the virtual machine snapshot is taken, the I/O is still directed to the base vVols object while the delta records the point-in-time (PIT) copy of the data. To achieve this, a new parameter called ddb.objectParentUri is added to the VMDK descriptor file on creation of the first snapshot. This line is the address of the base vVols object. The new string in the Extent description is the address of the snapshot file.

Descriptor file after snapshot:

[root@ESXi60] cat testVM.vmdk
# Disk DescriptorFile
version=4
encoding="UTF-8"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="no"
createType="vmfs"

# Extent description
RW 4194304 VMFS "vvol://1b38adb34b7f45f8-9abcb8e9eadba042/rfc4122.c74ae2ab-c908-407c-b629-cf7495cd98d5"

# The Disk Data Base
#DDB

ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "261"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "847f9654f70db75004bc90fbfffffffe"
ddb.nativeDeltaBytes = "2147483648"
ddb.objectParentUri = "vvol://1b38adb34b7f45f8-9abcb8e9eadba042/rfc4122.0a0a1e74-d7c4-4217-a49e-06281db2cffa"
ddb.uuid = "60 00 C2 91 c4 12 18 3b-5b 5c 70 bb 92 f6 9a e3"
ddb.virtualHWVersion = "11"

Note: The virtual machine configuration file (.vmx) file still appears to point to the delta virtual disk.

[root@ESXi60] grep -i scsi.*file *vmx
scsi0:0.fileName = "testVM-000001.vmdk"

On closer inspection, when you read the delta VMDK's descriptor file you will see that it references the ddb.objectParentUri of the base VMDK VVol object UUID. So, even though it appears to be running from the snapshot, the virtual machines is still running from the base VMDK disk. This is called re-parenting, also known as swizzling.

Whenever a snapshot is consolidated, the related snapshot file is discarded, the ddb.objectParentUri line is removed and the base descriptor file reverts to point to the original VVol UUID (the Extent description line returns to it's original value).

[root@ESXi60] cat testVM-000001.vmdk
# Disk DescriptorFile
version=4
encoding="UTF-8"
CID=fffffffe
parentCID=ffffffff
isNativeSnapshot="yes"
createType="vmfs"

# Extent description
RW 4194304 VMFS "vvol://1b38adb34b7f45f8-9abcb8e9eadba042/rfc4122.0a0a1e74-d7c4-4217-a49e-06281db2cffa"

# The Disk Data Base
#DDB

ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "261"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "847f9654f70db75004bc90fbfffffffe"
ddb.nativeParentCID = "fffffffe"
ddb.nativeParentHint = "testVM.vmdk"
ddb.uuid = "60 00 C2 91 c4 12 18 3b-5b 5c 70 bb 92 f6 9a e3"
ddb.virtualHWVersion = "11"

Note: The delta VMDK descriptor also contains a new ParentCID and ParentHint parameter, you will notice that every vVols snapshot is a child of the base disk.

This is because every snapshot is a point-in-time representation of the base disk. As such every snapshot will reference back to the base disk.

ddb.nativeParentCID = "fffffffe"
ddb.nativeParentHint = "testVM.vmdk"