Understanding vSAN on-disk format versions and compatibility
search cancel

Understanding vSAN on-disk format versions and compatibility

book

Article ID: 327034

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

This article outlines all vSAN on-disk format versions, including compatible features, interoperability and alternate version numbers.
 
vSAN has several different on-disk format versions available depending on the version and upgrade history of the cluster. Some on-disk format versions are transient while some are intended for long-term production. As certain vSAN features are tied to the on-disk format version, the format version must be accounted for when determining interoperability.

Environment

VMware vSAN 8.x
VMware vSAN 7.x
VMware vSAN 6.x
VMware vSAN 5.x

Resolution

vSAN on-disk format versions are reported differently in some vSAN releases.

vSAN on-disk format version reporting

Version per vSphere Web Client Version per RVC * Version per esxcli ** Version per CMMDS vSAN Release Notes
1 1 1 NA 5.5 ---
2 2 2 2 6.0-6.1 Format introduced with 6.0, includes performance and snapshot enhancements
2.5 2.5 2.5 3 6.2 Intermediate on-disk format to facilitate upgrade
3 3 3 4 6.2 Format introduced with 6.2, includes support for checksum, deduplication and compression
5 5 5 5 6.6 Format introduced with 6.6, includes support for encryption
6 6 6 6 6.7 Format introduced with 6.7, includes performance enhancements
7 7 7 7 6.7 U1 Format introduced with 6.7 U1
10 10 10 10 6.7 U3 Format introduced with 6.7 U3
11 11 11 11 7.0 Format introduced with 7.0
13 13 13 13 7.0 U1 Format introduced with 7.0 U1
14 14 14 14 7.0 U 2 Format introduced with 7.0 U2
15 15 15 15 7.0 U3 Format introduced with 7.0 U3
17 17 17 17 8.0 Format introduced with 8.0
18 18 18 18 8.0 U1 Format introduced with 8.0 U1
19 19 19 19 8.0 U2 Format introduced with 8.0 U2
20 20 20 20 8.0 U3 Format introduced with 8.0 U3

Notes:

  • * If RVC is an older version than vCenter, RVC will report the same as CMMDS.
  • ** ESXCLI does not report version information prior to vSAN 6.1.

vSAN on-disk format version support matrix

For interoperability, vSAN releases support the on-disk format versions of all prior releases,* and support is cumulative:
 
vSphere Version
On-disk format versions supported    
10 11 13 14 15 17 18 19 20
5.5, Update 1+ X X X X X X X X X X X X X X
6.0 GA
 6.0/6.2 U1 U2 U3
X X X X X X X X X X X X X
 
6.5 GA
X X X X X X X X X X X X X X
6.5.0d (vSAN 6.6)  X X X X X X X X X X X X X
6.7 X X X X X X X X X X X X
6.7 U1 X X X X X X X X X X X
6.7 U3 X X X X X X X X X X
7.0 GA X X X X X X X X X
7.0  U1 X X X X X X X X
7.0 U2 X X X X X X X
7.0 U3 X X X X X X
8.0 X X X X X
8.0 U1 X X X X
8.0 U2 X X X
8.0 U3 X X
"" denotes compatibility
 
 

Note for upgrade : The on-disk format upgrade from 2.0 to 3.0 will initiate two tasks:
A. 1MB Object Alignment
B. vSAN Object Conversion based on target version

After upgrading the disk groups to 3.0, only metadata upgrades are required for all future on-disk format upgrades. This does not require a full data evacuation. 

Before upgrading on-disk format versions, refer to the following KB for known issues surrounding clusters mid ESXi upgrade (Failure to promote CMMDs version resulting in vSAN cluster to become partitioned during update to vSAN 6.7 U1/U2/U3, vSAN 7.0)  
 
Important: vSAN release interoperability is dictated by the on-disk format version.
All hosts must be using the same on-disk format version or the cluster will run into operation issues as outlined below:
  • Potential cluster partitions
  • Data unavailability due to cluster partition 
  • Some disks/disk groups may fill while others don't 
  • Data Migration between hosts will be impacted 
For example If there is a four node vSAN cluster where two hosts are on ESXi version 6.5U3 and two 6.7U3, the highest supported version for 6.5U3 is on-disk format version 5.0 where as 6.7U3 may be on version 10 if not formatted with a lower on-disk format version as per KB How to format vSAN Disk Groups with a legacy format version (2146221).

Even though the 6.7 U3 hosts can support version 5.0 this is only for reads NOT writes. This means the hosts running ESXi version 6.7 U3 will only write to the disks with on-disk format version 10 not 5 resulting in an unbalanced disk utilization which can lead to some disks filling up while others have plenty of free space. All hosts, must be running the lowest common denominator of on-disk format in this example will be version 5.

Our recommendation would be to upgrade the ESXi hosts to the latest supported version and the latest consistent on-disk format version.

For more information about vSAN interoperability and upgrades, see VMware vSAN Upgrade Best Practices (2146381). To maintain compatibility, it may be necessary to format disks with an older on-disk format version. For more information, see How to format vSAN Disk Groups with a legacy format version (2146221).

Additional Information