VM on dedicated datastore reports intermittent latency under low load

Products

• VMware vSphere ESXi

Issue/Introduction

• A Virtual Machine has a dedicated datastore not shared with any other VM.
  
  
• Under low to moderate load conditions, there are intermittent I/O latency spikes on VM disks.
  
  
• The latency spikes persist after other VM optimizations are implemented (e.g. ensuring sufficient CPU and memory, distributing disks across multiple pvscsi SCSI controllers)
• ESXi host drivers and firmware for NICs/HBAs used for storage are supported and upto date as per the Broadcom Compatibility Guide.

Environment

VMware vSphere ESXi (all versions)

Cause

Intermittent I/O latency either at the storage array or network/fabric level.


To confirm, when issue is occuring, run:

esxtop

Type 'u' to select device view and confirm DAVG/cmd metrics.

For further informationon interpreting esxtop data, see: Using esxtop to identify storage performance issues for ESXi.

Resolution

Investigate the cause of the latency spikes on the storage array and network/fabric, and resolve.