Recommendation for ESXi NVMe/TCP Configuration for Hitachi Storage Array

Products

• VMware vSphere ESXi 8.0

Issue/Introduction

• Hitachi's NVMe/TCP storage system allows many NVMe subsystems (subsystem in short) to be created
• In the extreme case, different NVMe namespaces (namespace in short) presented to an NVMe host (host in short) can be exported through different subsystems.
• When a large number of namespaces from a Hitachi storage system needs to be presented to a ESXi host through different initiator ports and different target ports, this results in the total number of NVMe controllers (a controller is an association between a ESXi host and a subsystem through an initiator port and a target port)
• When there is an environment outage and the target down happens, which makes many controllers enter offline state, ESXi NVMe core layer starts the error recovery.
• This model can easily exhaust some type of system resources used by in the error recovery

Environment

• VMware ESXi 8.0U3 (and above) and Hitachi NVMe/TCP Storage System

Cause

• Due to a target port being down/offline, a large number of controllers went offline and entered recovery mode which exhausted the system resources. due to this some controllers may never be recovered

Resolution

• Hitachi's NVMe/TCP storage system also allows multiple namespaces exporting to a ESXi host through one subsystem
• With this, VMware's recommendation is to limit the maximum number of subsystems an ESXi host can access no more than 8, no matter how many namespaces that host needs to access