vSAN support of NVMe devices behind tri-mode controllers

Products

• VMware vSAN

Issue/Introduction

To provide guidance regarding supported NVMe devices in vSAN environments.

What is a tri-mode controller?

Discrete RAID controllers that support SATA/SAS/NVMe are often known as "Tri-mode controllers". 

VMware vSAN will only support SAS and SATA devices attached to a Tri-mode controller.
NVMe devices attached to Tri-mode controller are NOT a vSAN supported configuration.

Resolution

NVMe devices are only supported directly connected to a PCIe slot on the bus. 

 

Please note:

• NVMe devices attached to Tri-mode controller are NOT a vSAN supported configuration. This applies to storage controllers in RAID, HBA, Pass-through or JBOD mode.
• NVMe devices are only supported directly connected to a PCIe slot on the bus. Please work with your server vendor to replace drive backplanes, remove storage controllers or re-cable the system to achieve this configuration.

Reference: VMware vSAN Design Guide p.26

Tri-mode controllers
Discrete RAID controllers that support SATA/SAS/NVMe are often known as "Tri-mode controllers". While some of these devices may become certified on the vSAN VCG, they are not supported for use with NVMe devices attached to them and passing IO through them. Tri-Mode controllers may only be used with SAS and SATA devices. NVMe drives are expected to connect to PCI-E
without passing through a RAID controller. In cases where additional PCI-E lanes are needed to support dense server configurations, PCI Switches are a supported alternative.

Additional Information

Work Around:

contacting server vendor to have appropriate hardware change to include PCIe slots for the NVMe devices.

1.  Place host into maintenance mode using Ensure Accessibility
2.  Use Disk Management GUI to decommission the diskgroup(s) on the target host.
3.  Shut down the hardware and reconfigure the server so the NVMe drives are connected to the PCI bus as intended.
4.  Bring the server up and recreate the diskgroup(s) through disk management.
5.  Repeat steps 1-4 with remaining hosts in the cluster