vSAN support of NVMe devices behind tri-mode controllers
search cancel

vSAN support of NVMe devices behind tri-mode controllers

book

Article ID: 314305

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

To provide guidance regarding supported NVMe devices in vSAN environments. This article is applicable for both OSA and ESA configurations.

What is a tri-mode controller?

Discrete RAID controllers that support SATA/SAS/NVMe are often known as "Tri-mode controllers". 

VMware vSAN will only support SAS and SATA devices attached to a Tri-mode controller.
NVMe devices attached to Tri-mode controller are NOT a vSAN supported configuration.

Symptoms of running in this configuration can include:
  • Severely reduced performance(Ie latency)
  • Component checksum errors
  • Disk groups or storage pools going offline
  • Data unavailability 
  • Incompatible disks

Environment

  • VMware vSAN 7.x
  • VMware vSAN 8.x
  • VMware vSAN 9.x 

Resolution

NVMe devices are only supported directly connected to a PCIe slot on the bus. 
 

Please note:

  • NVMe devices attached to Tri-mode controller are NOT a vSAN supported configuration. This applies to storage controllers in RAID, HBA, Pass-through or JBOD mode.
  • NVMe devices are only supported directly connected to a PCIe slot on the bus. Please work with your server vendor to replace drive backplanes, remove storage controllers or re-cable the system to achieve this configuration.

Reference: VMware vSAN Design Guide , under the Choosing a Storage I/O Controller section.

Tri-mode controllers
Discrete RAID controllers that support SATA/SAS/NVMe are often known as "Tri-mode controllers". While some of these devices may become certified on the vSAN VCG, they are not supported for use with NVMe devices attached to them and passing IO through them. Tri-Mode controllers may only be used with SAS and SATA devices. NVMe drives are expected to connect to PCI-E
without passing through a RAID controller. In cases where additional PCI-E lanes are needed to support dense server configurations, PCI Switches are a supported alternative.
 

To resolve the issue,contact the server vendor to have appropriate hardware change to include PCIe slots for the NVMe devices. After that, proceed with the steps as below: 

  1. Place host into maintenance mode using Ensure Accessibility
  2. Use Disk Management GUI to decommission the diskgroup(s) on the target host.
  3. Shut down the hardware and reconfigure the server so the NVMe drives are connected to the PCI bus as intended.
  4. Bring the server up and recreate the diskgroup(s) through disk management.
  5. Repeat steps 1-4 with remaining hosts in the cluster

Additional Information

Powered by Wolken Software