Tuning ESXi for better storage performance by modifying the maximum I/O block size

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides steps to reduce the size of I/O requests passed to the storage device to tune and optimize storage performance for ESX/ESXi.

Resolution

Many applications are designed to issue large I/O requests for higher bandwidth. ESXi pass I/O requests as large as 32767 KB directly to the storage device. I/O requests larger than this are split into several, smaller-sized I/O requests.

 

Some storage devices, however, have been found to exhibit reduced performance when passed large I/O requests (above 128KB, 256KB, or 512KB, depending on the array and configuration). As a fix for this, you can lower the maximum I/O size ESX allows before splitting I/O requests.

If you have measured a decreased storage performance in ESX/ESXi 3.5 and later hosts, compared to a similar ESX 3.0.x system, try reducing the maximum I/O size, as described below, and see if performance improves. If your storage device does not have this problem (or if the problem does not go away when you reduce the maximum I/O size), you are better off leaving the maximum I/O size at its default 32767 KB setting because it increases performance and (or) lower CPU utilization on your system.

One way to diagnose the problem is by looking at latency statistics reported by esxtop. Esxtop includes several detailed storage statistics that report time spent in various components. If storage devices are a problem, esxtop displays high device latencies. For more information about using esxtop, see Using esxtop to identify storage performance issues.

To reduce the size of I/O requests passed to the storage device using the vSphere Client:

• Go to Host > Configuration. 
• Click Advanced Settings.
• Go to Disk.
• Change Disk.DiskMaxIOSize.

To check the size of I/O requests passed to the storage device using the command line:

esxcli system settings advanced list -o /Disk/DiskMaxIOSize
   Path: /Disk/DiskMaxIOSize
   Type: integer
   Int Value: 32767
   Default Int Value: 32767
   Min Value: 32
   Max Value: 32767
   String Value:
   Default String Value:
   Valid Characters:
   Description: Max Disk READ/WRITE I/O size before splitting (in KB)
   Host Specific: false
   Impact: none

 

Split commands can been seen from the device section of esxtop, to collect performance stats please see : Collecting esxtop batch data for ESXi performance troubleshooting

 

Note: You can make this change without rebooting the ESX/ESXi host or without putting the ESX/ESXi host in maintenance mode.

Other related information when this setting is changed - Virtual machines using EFI firmware fails to start from certain storage devices