Setting the Maximum Outstanding Disk Requests for virtual machines

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

You can adjust the maximum number of outstanding disk requests with the Disk.SchedNumReqOutstanding parameter. When two or more virtual disks share a LUN (logical unit number), this parameter controls the total number of outstanding commands permitted from all virtual disks collectively on the host to that LUN (this setting is not per virtual machine).

Notes:

• This limit does not apply when only one virtual machine with single virtual disk is active on a datastore/LUN. In that case, the bandwidth is limited by the queue depth of the storage adapter.
• The per-host parameter Disk.SchedNumReqOutstanding is deprecated in vSphere 5.5. The setting is now per device/LUN. For more information, see the Solution section in this article.

This change can impact disk bandwidth scheduling, but experiments have shown improvements for disk-intensive workloads. If you adjust this value in the VMkernel, you might also want to adjust the queue depth in your storage adapter.

For more information on how to complete this action for a QLogic qla2x00 Fibre Channel adapter, see Changing the Queue Depth for QLogic and Emulex HBAs (1267).

Environment

VMware vSphere ESXi 6.5.x
VMware vSphere ESXi 6.7
VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 8.0.x

Resolution

Notes:

• These procedures only apply to the ESXi host that the parameters are changed on. You must make the same changes to all other ESXi hosts that have the datastore/LUN presented to them.
• Ensure that the current value of any parameters are recorded before changing them in case you wish to revert back to their default values.

Warning: This change can have a major impact on the performance of your storage array. Use extreme caution and only adjust this parameter on the recommendation of your storage vendor.

vSphere 5.5 and later

To set the VMkernel limit for a particular LUN/device in vSphere 5.5 and later using the command line:

1. Open an SSH session to the host
2. To check the current value for a device,run the command:
   
   esxcli storage core device list -d naa.xxx
   
   Note: The value appears under No of outstanding IOs with competing worlds:
    
3. To modify the current value for a device,run the command
   
   esxcli storage core device set -d naa.xxx -O Value
   
   Where Value is between 1 and 256 (vSphere 5.5 and 6.0) and between 1 and Max Queue depth of the device (vSphere 6.5 and later)

 

Note:In vSphere ESXi 6.5 and later, the No. of oustanding IOs with competing worlds parameter is limited to Max Queue Depth of the device as the host should not be queueing more than the allowed device queue depth as this could lead to command failures.