VMs experiencing high Read/write latency on vSAN Datastore

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• One or more Virtual Machines are experiencing high read and write latency on a vSAN datastore.
• Latency remains consistently around 50–100 ms, which is significantly above normal operational thresholds.
• Application performance within affected VMs is degraded or becomes unresponsive due to delayed read/write operations.
• Copying large files (e.g., ~100 GB) between VMs takes several hours to complete.
• The vSAN network is configured with 1 Gbps NICs.
• vSAN performance graphs show that throughput consistently reaches the maximum 1 Gbps limit, with latency spiking at the same time.
  
   

Environment

VMware vSAN 7.x
VMware vSAN 8.x

Cause

The performance degradation is is due to network latency and bandwidth saturation. The throughput consistently reaches ~95 MB/s, which is the theoretical limit of a 1 Gbps network.
Since vSAN operations rely heavily on network throughput for data replication, object management, and resync tasks, the limited bandwidth results in slow task completion and degraded VM performance.

 

Resolution

• Upgrading to 10 Gbps network interfaces allows vSAN to efficiently handle higher I/O workloads, reduces queuing, improves VM responsiveness, and ensures reliable performance for critical operations such as replication, resync, and rebuild tasks.
• This aligns with VMware vSAN best practices, which mandate 10 Gbps or higher network bandwidth for production deployments to ensure stable and optimal performance.