Storage vMotion Performance Degradation Due to RAID Configuration on local NVMe Datastores

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms

- Storage vMotion tasks (both cold and hot) involving a specific ESXi host take twice as long as expected.
- Network bandwidth utilization during vMotion is significantly lower than expected
- Performance degradation is only observed when RAID is enabled on the locally attached NVMe datastore.
- Disabling RAID on the NVMe datastore restored expected performance, allowing full network bandwidth utilization during vMotion task .

Environment

vSphere 8.x

Cause

This issue is linked to how RAID volumes with 4k-native sector sizes are handled in ESXi versions prior to 9.0.

When a RAID volume is presented as a 4k-native HDD, ESXi emulates it as a 512e disk.
This emulation triggers read-modify-write (RMW) operations in the ESXi storage stack.
RMW operations incur significant CPU overhead, which can throttle performance during storage-intensive tasks like vMotion and snapshot deletion.

Resolution

Option 1: Upgrade to ESXi 9.0

Upgrade the affected host(s) to ESXi 9.0.
Rebuild the RAID volume after the upgrade to ensure it is presented as a 4kn SSD.
This eliminates the need for RMW operations and restores full performance.

Important: Simply upgrading without rebuilding the RAID volume will not resolve the issue. The volume must be recreated to benefit from improved sector format handling.

Option 2: Remain on ESXi 8.x

If upgrading is not feasible, consider disabling RAID on the NVMe datastore.
This avoids the 4k emulation and allows full network bandwidth utilization.

Additional Information

For details on supported sector formats in ESXi 9.0, refer to:
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/9-0/vsphere-storage/managing-esxi-storage-devices/device-sector-formats.html