Poor performance and high disk latency with some storage configurations

Products

• VMware vSphere ESXi

Issue/Introduction

• Poor virtual machine performance
• Poor disk latency during backups
• Virtual disk latency

Resolution

This issue may occur if your virtual machines do not have enough I/O per second (IOPS) and IOPS available per virtual machine is less than 30.

Calculating the IOPS available per virtual machine

Each disk in the RAID volume backing your LUNs is capable of a certain amount of throughput. For a 7200 RPM (7k) drive, you get approximately 100 IOPS per disk, and the I/O per disk increases as the spindle speed increases. The IOPS per disk vary depending on the RAID type used when configuring your array and/or LUN. For more information, contact your storage vendor.

These numbers indicate the IOPS that is possible using the VMFS file system, which is optimized to avoid hot spots on the disk and is not optimized for raw access speed:

Disk Spindle Speed (RPM)	IOPS available
7200 RPM	100
10,000 RPM	150
15,000 RPM	230

 

Calculating the total IOPS per virtual machine

To calculate the total IOPS per virtual machine, find the speed of your drives and determine their IOPS. Each drive in your RAID volume within your SAN contributes to the total I/O available for that volume. IOPS per virtual machine is calculated by dividing the total capacity available by the number of virtual machines running on that volume (which may contain more than one LUN).

For example, if you have six 10,000 RPM drives, the total IOPS available in that volume is 150 x 6 = 900. If you are running 50 virtual machines on LUNs located on this RAID volume, each virtual machine gets an average of 900 / 50 = 18 IOPS. In this case, you experience relatively poor virtual machine performance.

In this example, to obtain acceptable performance you should not exceed 900 / 30 = 30 virtual machines on that volume.

Note: Backing up the storage consumes more IOPS and adds an additional strain on the volume. In this case, you may want to have an additional overhead capacity to handle this. To determine the amount of additional capacity you may require to perform backups, contact your backup vendor.