VMware Storage vMotion fails at 21% with the error: Storage VMotion failed to copy one or more of the VM's disks

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Cannot Storage migrate a virtual machine between two LUNs.
• Performing a Storage vMotion migration of a virtual machine fails at 21%
• You see the error:
  
  A general system error occurred: Storage vMotion failed to copy one or more of the VM's disks. Please consult the VM's log for more details, looking for lines with "svMotion"
  
  'Failed to copy one or more disks. Cancelling Storage vMotion.
  Cannot copy source(/vmfs/volumes/uuid/vmname/vmname.vmdk) to destination disk(/vmfs/volumes/uuid/vmname/vmname.vmdk)
  DM copy operation failed to copy 1 extents (Error:IO was aborted). Bytes transferred:0 Extents transferred: 0
  DM copy operation failed to copy 1 extents (Error:IO was aborted). Bytes transferred:0 Extents transferred: 0
   
• This error is reported irrespective of the LUN selected on the target array.
• In the /var/log/vmkernel.log file, you see:
  
  YYYY-MM-DDTHH:MM:SS.563Z cpu3:4936442)NMP: nmp_ThrottleLogForDevice:2318: cmd 0x83 (0x412440ba41c0, 4936839) to dev "naa.60060e80101aea40058bae7400000012" on path "vmhba4:C0:T4:L89" Failed: H:0x2 D:0x2 P:0x0 Possible sense data: 0xa 0xd 0x2. Act:EVAL

Environment

VMware vSphere ESXi 5.0
VMware ESXi 4.0.x Embedded
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 5.5
VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.1

Cause

This issue occurs if VAAI is enabled on the ESXi host, and the connected storage array does not execute VAAI commands correctly or successfully. This can be caused by out-of-date firmware on the storage array, or an unsupported make and model of array. The indicated SCSI return code shows an XCOPY command failed due to a Bus Busy condition. If the storage array did not support VAAI, the host would not have tried to use the XCOPY command, so the command itself failed when it should have succeeded.

Note: When a storage connection splitter is used, the splitter itself may fail to recognize VAAI commands or deal with them correctly.

Resolution

The correct resolution of this issue is to engage the storage vendor and have them investigate the XCOPY failures and resolve them. This may require a maintenance window if the array needs to be rebooted.

The accepted temporary workaround for this issue is to disable VAAI on the ESXi host while performing a Storage vMotion migration.
 

For more information, see Disabling the VAAI functionality in ESXi/ESX (318224).

Additional Information

Impact/Risks:
This error will affect both manual and SDRS storage vMotions; both will fail similarly. In general the use of VAAI operations yield large performance increases for certain operations which otherwise would be time-consuming. Turning off VAAI will cause performance loss in a healthy environment.