vSAN support for large capacity drives

Products

• VMware vSAN

Issue/Introduction

This article outlines vSAN support for large capacity drives in vSAN 7.x & 8.x OSA and higher environments.

Environment

VMware vSAN 7.x

VMware vSAN 8.x OSA

Cause

• vSAN prior to release 7.x supported only 32 bit disk group layouts.
• This layout only supports 16TB logical capacity capacity tier disks regardless if the physical disk is larger than 16TB.
• vSAN after release 7.x automatically uses 64 bit layout on disk groups with at least one large capacity drive. Existing disk groups which were created in vSAN versions prior to 7.x are not upgraded to 64 bit layouts after upgrading to 7.x or higher.
• Upgrading the on-disk format to version 11 or higher won't upgrade to 64 bit either.

Large capacity drives are defined as any capacity tier disk larger than 16TB.

Resolution

• One option is to continue to use the disk group without any modifications. This is fully supported. Release 7.x and higher supports 32 bit layouts regardless of the drive size. The only drawback is logical capacity of the capacity drive is limited to 16TB. 
  • This means for a disk group with 3x20TB capacity tier disks the logical capacity for that disk group will only be 48TB not 60TB which means you lose 12TB of capacity
• If you want the full capacity of your larger capacity tier disks then you will need to recreate the disk groups one at a time so they're recreated as 64 bit giving you complete access to the disks.

To identify the existing disk group layout, use the vdq commands on the ESXi host:

1) To see the disk group mappings and to get the device names run vdq -iH

  [root@esxi3:~] vdq -iH
Mappings:
   DiskMapping[0]:
           SSD:  mpx.vmhba2:C0:T0:L0
            MD:  mpx.vmhba2:C0:T1:L0

2) To see individual disk info run vdq -s -d <device>

VSAN_Metadata:
        checksum: 0x5509870ede2846c
         version: 0x4
     numMappedDisks: 1
          SSD:1: 0
      dedupScope:2: 2
       diskState:3: 0
      isAllFlash:1: 1
    diskFormatVersion: 12
     diskgroupLayout: 64 bit
        diskUUID: 523470d9-b0b7-f6fc-52be-############
        hostUUID: 5e83ca04-bac6-202f-1e79-############
       clusterUUID: 00000000-0000-0000-0000-000000000000
       encrypted:1: 0
      diskTable[00]: 52c32a88-cce8-fbc0-2e52-############

The diskgroupLayout displays the layout. A 32bit layout indicates 16TB logical space limited capacity tier disks for disk groups.