vSphere and vSAN support for Hot-plug of NVMe SSDs on AMD EPYC Processors

Products

• VMware vSAN
• VMware vSphere ESXi

Issue/Introduction

vSphere ESXi has PCIe Native Hot-plug support for NVMe SSDs on some limited platforms that meet vSphere special firmware requirements in addition to conforming to PCI Express Base Specification and Standard Hot-Plug Controller and Subsystem Specification. 
 
This article describes the additional requirements to support vSAN hot-plug configuration of NVMe SSDs on AMD EPYC Processors.

Environment

VMware vSphere 6.7.x

Resolution

As a workaround to support NVMe hot-plug on AMD EPYC processors:

1. Set kernel boot options as follows. The options require a reboot to take effect.

esxcli system settings kernel set -s dmaMapperPolicy -v disabled
esxcli system settings kernel set -s iovDisableIR -v true

2. Do not attempt to pass a hot-plugged device to a VM using the VMDirectPath I/O feature. See VMware vSphere VMDirectPath I/O: Requirements for Platforms and Devices (2142307)

For more information and platforms supported, please refer to VMware Compatibility Guide for vSAN.

Terms:

• Hot-Add/Hot-Remove: Adding or removing a device to/from a running system
• Hot-Plug: A generic term for used for hot-add or hot-removal of devices
• PCIe Native Hot-plug: PCI Express Hot-plug architecture designed to natively support hot-plug as per the PCIe Base Specification