Glossary of vSAN Terms and Acronyms

Products

• VMware vSAN

Issue/Introduction

This article outlines some of the common acronyms, terms and concepts used in vSAN.

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Resolution

Types of vSAN architecture:

• vSAN Original Storage Architecture (OSA):

vSAN Original Storage Architecture (OSA) is designed for a wide range of storage devices, including flash solid state drives (SSD) and magnetic disk drives (HDD). Each host that contributes storage contains one or more disk groups. Each disk group contains one flash cache device and one or more capacity devices (SSD or HDD).

• vSAN Express Storage Architecture (ESA):

vSAN Express Storage Architecture (ESA) is designed for high-performance NVMe based TLC flash devices and high performance networks. Each host that contributes storage contains a single storage pool of four or more flash devices. Each flash device provides caching and capacity to the cluster.

vSAN Cluster Deployment Options:

This section covers the different supported deployment options that are supported for vSAN clusters.

 

• Standard vSAN Cluster:

A standard vSAN cluster consists of a minimum of three hosts. Typically, all hosts in a standard vSAN cluster reside at the same location, and are connected on the same Layer 2 network. All-flash configurations require 10 Gb network connections, and this also is recommended for hybrid configurations.

• Two Host vSAN Cluster

Two host vSAN clusters are often used for remote office/branch office environments, typically running a small number of workloads that require high availability. A two host vSAN cluster consists of two hosts at the same location, connected to the same network switch or directly connected. You can configure a two host vSAN cluster that uses a third host as a witness, which can be located remotely from the branch office. Usually the witness resides at the main site, along with the vCenter Server.

• vSAN Stretched Cluster

A vSAN stretched cluster provides resiliency against the loss of an entire site. The hosts in a stretched cluster are distributed evenly across two sites. The two sites must have a network latency of no more than five milliseconds (5 ms). A vSAN witness host resides at a third site to provide the witness function. The witness also acts as tie-breaker in scenarios where a network partition occurs between the two data sites. Only metadata such as witness components is stored on the witness.

 

vSAN object terminology

Some common terms associated with vSAN objects are:

• Object (also called: DOM Object) - The final, usable storage space in vSAN. All vSAN data is stored in objects. There are three types of objects (discussed below), used for different purposes. All objects are comprised of one or more components and witnesses.

• Component (also called: LSOM Object) - A piece of an object. Components are a stripe or a mirror where object data are stored.

• Witness - Also a piece of an object (a component sub-type). Witnesses are a pure metadata (no production data) that act as a voter to maintain object quorum. Witnesses are associated to a single object and act as the tiebreaker/king-maker in the event of a network partition to avoid a split-brain scenario (e.g., to restrict access to the object to a single side of the network partition). The witness helps to maintain rule-based object availability. 

• Quorum - The availability requirement that dictates whether or not a vSAN object can be activated/used. In vSAN, quorum is defined as:

  greater than 50% (>50%) component/witness availability, with at least one complete, intact mirror available.

 

vSAN object types

The vSAN object types are:

• vmnamespace object - These objects are used for virtual machine working directory (The VM Home object type in the Web Client). These objects are formatted with VMFS and used for the hosting of small configuration files and VMX core dumps. The policy for these objects is always thin-provisioned. The VMFS filesystems on these objects are mounted via OSFS for access by the hosts. These objects are always FTT=1/SW=1/PC=0/FP=0, as defined in the cluster-wide default policy.

• vswp object - These objects are used for virtual machine swap space. They are not revealed in the Web Client, though there is a vmswap object in existence for all powered-on virtual machines whose home directories are a vSAN namespace (unless swap has been manually redirected). The policy for these objects is always FTT=1/SW=1/PC=100/FP=1, as defined in the cluster-wide default policy.

• vdisk object - These objects are used for virtual machine disks (VMDKs). In vSAN, the vdisk object type exists in lieu of the traditional -flat.vmdk (base disk) and -delta.vmdk (snapshot disk) files like those seen with VMFS and NFS datastores. The cluster-wide default policy for these objects is FTT=1/SW=1/PC=0/FP=0. The policies can be adjusted on a per-object basis using SPBM.

• Memory object Created when the snapshot memory option is selected when creating or suspending a virtual machine.

 

Storage policy common terms

Some common terms associated with storage policies are:

 

Site disaster tolerance:

 

Defines the data redundancy method used by stretched clusters to handle a site failure. This attribute applies to stretched clusters. If you have a standard vSAN cluster, choose None (standard cluster).The options are:

• None (standard cluster)

• Dual-site mirroring (stretched cluster)

• None - Keep data on primary (stretched cluster)

• None - Keep data on secondary (stretched cluster)

Failures to tolerate:

Defines the number of host and device failures that a virtual machine can tolerate. You can choose to have no data redundancy, or select a RAID configuration optimized for either performance (Mirroring) or capacity (Erasure Coding).

• RAID-1 uses more disk space but provides better performance.

• RAID-5/6 (Erasure Coding) uses less disk space, but the performance is reduced in vSAN OSA clusters.

Table 1. RAID Configurations, FTT, and Host Requirements

RAID Configuration	Failures to Tolerate (FTT)	Minimum Hosts Required
RAID-1 (Mirroring)	1	2
RAID-5 (Erasure Coding)	1	4
RAID-1 (Mirroring)	2	5
RAID-6 (Erasure Coding)	2	6
RAID-1 (Mirroring)	3	7

 

Advanced Attributes for vSAN VM Storage Policies

 

Number of disk stripes per object:

Defines the minimum number of storage devices across which each replica of a virtual machine object is striped. A value higher than 1 might result in better performance, but also makes more use of system resources. Default value is 1. Maximum value is 12. Change the default value only when recommended by VMware support.

IOPS limit for object

Defines the IOPS limit for an object, such as a VMDK. IOPS is calculated as the number of I/O operations, using a weighted size. If the system uses the default base size of 32 KB, a 64-KB I/O represents two I/O operations.

When calculating IOPS, read and write are considered equivalent, but cache hit ratio and sequentially are not considered. If a disk’s IOPS exceeds the limit, I/O operations are throttled. If the IOPS limit for object is set to 0, IOPS limits are not enforced.

vSAN allows the object to double the rate of the IOPS limit during the first second of operation or after a period of inactivity.

Object space reservation

This setting defines the percentage of the logical size of the virtual machine disk (vmdk) object that must be reserved (provisioned) when deploying virtual machines. The default reservation value in VMware Cloud on AWS is 0% (Thin provisioning). You can specify Thick provisioning to reserve capacity for larger-than-expected vSAN writes, but the underlying vmdk structure remains the same as it is in the Thin provisioning configuration, and is not the same as the Thick provision eager zeroed provisioning model available on-premises.

As noted in Storage Resources, you should consider setting the Object Space Reservation (OSR) advanced policy setting to Thin provisioning. OSR controls only the space reservation and has no performance impact. Although capacity management is often critical for on-premises data centers, VMware Cloud on AWS Elastic DRS ensures the cluster will not run out of free space.

Flash read cache reservation

This setting is ignored in VMware Cloud on AWS. In Hybrid vSAN deployments, it designates how much flash capacity is reserved as read cache.

Disable object checksum

If the option is set to No, the object calculates checksum information to ensure the integrity of its data. If this option is set to Yes, the object does not calculate checksum information.

vSAN uses end-to-end checksum to ensure the integrity of data by confirming that each copy of a file is exactly the same as the source file. The system checks the validity of the data during read/write operations, and if an error is detected, vSAN repairs the data or reports the error.

If a checksum mismatch is detected, vSAN automatically repairs the data by overwriting the incorrect data with the correct data. Checksum calculation and error-correction are performed as background operations.

The default setting for all objects in the cluster is No, which means that checksum is enabled.

Force provisioning

If the option is set to Yes, the object is provisioned even if the Primary level of failures to tolerate, Number of disk stripes per object, and Flash read cache reservation policies specified in the storage policy cannot be satisfied by the datastore. Use this parameter in bootstrapping scenarios and during an outage when standard provisioning is no longer possible.

The default No is acceptable for most production environments. vSAN fails to provision a virtual machine when the policy requirements are not met, but it successfully creates the user-defined storage policy.

 

vSAN Architecture Terms:

Some common acronyms/terms associated with vSAN architecture are:

• CLOM - Cluster-Level Object Manager. Responsible for object placement in such a way that satisfies the defined policy. Handles initial provisioning and cluster balancing, object reconfiguration, etc.

• DOM - Distributed Object Manager. This function within vSAN keeps track of the various objects (VM directories, VMDKs, etc.) and assembles the objects based on their corresponding components.

• Fault Domain - One more vSAN servers that make up a failure group. The FTT policy option defines how many fault domain failures an object should survive. By default, each host is its own fault domain. Multiple hosts can be grouped into a single fault domain (for example, all hosts in one rack or one physical location) in vSAN 6.0+.

• LSOM - Local Log-Structured Object Manager. This function within vSAN keeps track of object components per-disk and provides persistence for object data, attributes, etc.

• MD - Magnetic Disk. Used for capacity tier in vSAN hybrid clusters.

• OSFS - Object Store File System. Provides a filesystem-like framework for interacting with vSAN objects* (provides the vsanDatastore container). Handles mounting and access to namespace objects.

  Note: OSFS is also used with VVOL.

• PLOG - Physical Disk Log. Helps with tracking blocks on MD devices and ensuring write coherence.

• SPBM - Storage Policy-Based Management. This is what enables vSAN to use the Profile-Driven Storage Service that runs with vCenter to validate, change and execute storage policies against vSAN objects (VMs).

• SSD - Solid State Disk. Used for cache tier (vSAN hybrid) or for both cache and bulk-capacity tiers (vSAN all-flash - assumes write-intensive/ultra-high-performance SSD for cache tier).

• VASA - vSphere APIs for Storage Awareness. This is a mechanism that allows a storage platform to connect to SPBM via "VASA provider" supplied by the storage vendor.

• vSAN - VMware vSAN. This is the name of the product.

• vsanvpd - vSAN VASA Provider Daemon. This is the host-resident storage provider that registers with SPBM to enable storage policies for vSAN objects. (Deprecated in versions 7.x and newer)