Low I/O performance using intel-nvme drivers 1.3.2.8-1OEM and 1.3.2.4-1OEM with block sizes larger than 4K

Products

• VMware vSphere ESXi

Issue/Introduction

To resolve the performance issue

Symptoms:

• Significant performance degradation is observed when I/O block size is greater than 4K.
• 4K block size IOPS are not affected.

Environment

VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.0

Resolution

This issue issue is resolved in driver version 1.4.0.1016-1OEM  for ESXi 6.5 and 6.7. 

To check the current version of driver, run

•  

In ESXi , run any of these  commands:      esxcli software vib list | less

    esxcli software vib list | egrep String
     For eg., esxcli software vib list | egrep intel-nvme

To download the driver for ESXi 6.7 go to MyVMware
For ESXi 6.5 go to MyVmware

Note: while ESXi 7.0 is not affected by this issue, if required you can download the driver for ESXi 7.0 from MyVMware as well.



For steps to install the driver refer to VMware KB : How to download and install async drivers in ESXi 5.x/6.x 

Workaround:
To workaround this issue downgrade to the intel-nvme driver 1.2.1.15-1OEM  for ESXi 6.7 and 6.5 and, for ESXi 6.0, use 1.2.0.9-1OEM.

To do this downgrade on hosts that have the SSH shell enabled:

1. Upload (FTP) the driver VIB to /tmp and execute the following command from the SSH command line:

​esxcli software vib install --force --no-sig-check --maintenance-mode

--viburl=file:/tmp/intel-nvme-1.2.1.15-1OEM.<rest of driver name>.vib

Note: For ESXi 6.0 use the VIB file: intel-nvme 1.2.0.9-1OEM

2. A host reboot is now required to activate the driver.

Additional Information

Impact/Risks:
Requires reboot of ESXi host.