vSAN Health Service - Cluster Health - Homogeneous vSAN disk sector size

Products

• VMware vSAN

Issue/Introduction

This article explains the Cluster health - Homogeneous vSAN disk sector size check in the vSAN Health Service and provides details on why it might report a warning/error and how to fix the warning/error state.

Starting from VCF 9.0 release, VSAN ESA can be created on 4KN NVMe devices. With addition of this support, there is a possibility that the ESXi host has a mix of 512N and 4KN NVMe devices, while this is a supported config but is not a recommended configuration. A recommendation is to always use homogenous devices for optimal performance

Environment

VMware vSAN 9.0 and higher

Resolution

Q: What does the Cluster Health – Homogeneous vSAN disk sector size check do?

It checks that all storage devices within the vSAN ESA cluster have a consistent sector size, either 512 or 4K bytes. This prevents mixed sector sizes in the cluster, ensuring optimal performance and reliability.

Q: What does it mean when it is in an error state?

When the health status is in an error state, it indicates that mixed sector sizes (512 and 4K bytes) of NVMe disks have been detected within the vSAN ESA cluster, excluding the witness host.

Q: How does one troubleshoot and fix the error state? 

To fix the error state from this health check, you can ungroup the disk from the vSAN storage pool and then choose to either reformat or replace it.

To reformat the disk sector size, follow these steps

1. Navigate to vSAN Cluster → Configuration → Disk Management, safely evacuate the data, and remove the disk from the storage pool.
2. Retrieve the namespaces connected to the host and view the namespace details using the command: /bin/esxcli nvme namespace list 
3. Check if the disk supports 4K native sectors and retrieve corresponding LBA format number by running the command /bin/esxcli nvme namespace identify -n <namespace name> . Below is an example of the command output.
   

   Name      Value                             Description --------  --------------------------------  ----------- ... LBAF0     0x90000                           LBA Format 0 Support LBAF1     0x90008                           LBA Format 1 Support LBAF2     0xc0000                           LBA Format 2 Support LBAF3     0xc0008                           LBA Format 3 Support LBAF4     0xc0040                           LBA Format 4 Support

   
   If the output shows 0xc0000, it indicates that the disk supports a 4K native sector size, corresponding to LBA format 2. Conversely, 0x9000 represents support for 512-byte sectors, mapped to LBA format 0. The format number will be used in Step 5
   
   
4. Get controller - vmhba mapping through command /bin/esxcli nvme controller list . Below is an example of the command output.
   

   Name                Controller Number  Adapter  Transport Type  Is Online  Controller Type  Is VVOL  Keep Alive Timeout  IO Queue Number  IO Queue Size ------------------  -----------------  -------  --------------  ---------  ---------------  -------  ------------------  ---------------  ------------- <Controller Name>                256   vmhba1    PCIe                 true                    false                   0                2           1024 <Controller Name>                257   vmhba0    PCIe                 true                    false                   0                2           1024

5. Reformat the disk to the desired sector size through command 
   /bin/esxcli nvme device namespace format -f <LBA format> -s 1 -n 1 -p 0 -A <vmhba name> -m 0 -l 0 
   
   Refer to the target <LBA format> number identified in step 3. This is an integer value. For example, LBAF0 corresponds to 0.

Additional Information

• FAQ: Support statement for 512e and 4K Native drives for VMware vSphere and vSAN
• vSAN 健全性サービス - クラスタの健全性 - 均一の vSAN ディスク セクター サイズ