Impact on VM performance when we perform Compute and storage vMotion with Virtual Disk IOPS - Limit on VM.
Article ID: 344984
Updated On:
Products
Issue/Introduction
You will notice xvMotion process taking longer time to complete and it may impact the virtual machine performance during the time of migration.
Compute and storage vMotion (also called as xvMotion) can do two different types of storage copy
a. Local storage copy: Source host has access to both source and destination datastore. Source host drives the copy without the involvement of the compute network and destination host.
b. Remote storage copy: Source doesn't have access to the destination datastore. So source host relies on the compute network to connect to destination ESX host which copies the data to destination datastore.
This issue is only seen for remote storage copy during compute and storage vMotion.
Sample log snippets from VMware.log
2023-03-16T04:44:19.017Z| vmx| | W003: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 0 MB at 0 kB/s.
2023-03-16T04:44:29.022Z| vmx| | W003: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 0 MB at 0 kB/s.
2023-03-16T04:44:39.025Z| vmx| | W003: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.00 seconds, copied total 0 MB at 0 kB/s.
2023-03-16T04:44:49.027Z| vmx| | W003: SVMotion: scsi0:0: Disk transfer rate slow: 6552 kB/s over the last 10.00 seconds, copied total 64 MB at 1310 kB/s.
2023-03-16T04:44:59.037Z| vmx| | W003: SVMotion: scsi0:0: Disk transfer rate slow: 0 kB/s over the last 10.01 seconds, copied total 64 MB at 1091 kB/s.
Environment
VMware vSphere ESXi 8.0.x
Cause
The remote storage copy is programmed to run at GB/sec speeds. Having an IOPS limit which is lower than this rate will limit the rate at which the remote storage copy runs leading to slower than usual rate of copying the disk. This would also mean that remote storage copy stream will dominate and take the majority of this IOPS limit, hardly leaving anything for the VM's workload I/O. So the VM performance is affected during remote storage copy.
So both the disk copy and the VM performance gets affected.
Resolution
VMware is aware of this issue and working on fixing it in the upcoming release. Please subscribe to the KB article to get alerted when the issue is fixed.
Workaround:
You could possible disable the IOPS limit on the disk when you perform Compute and Storage vMotion (also called as xvMotion).
Additional Information
1. Disk IOPS limit on VM would not impact storage vMotion.
2. Disk IOPS limit on VM would not impact compute and storage vMotion when storage copy is locally driven by the source ESX host without the involvement of destination ESX and compute network. In case of the local storage copy, it gets it owns scheduler queue without any limits.
Impact/Risks:
Feedback
