Storage vMotion failure due to backend storage issues
Article ID: 411186
Updated On:
Products
Issue/Introduction
Storage vMotion operation failed during the migration of VM from source host <host IP address> to destination host <host IP address>.
vmware.log:
[YYYY-MM-DDTHH:MM:SS]worker-#######- SVMotion: Enter Phase #
[YYYY-MM-DDTHH:MM:SS]In(05) worker-##### - Disk/File copy started for /vmfs/volumes/########/########/virtual machine.vmdk
......
[YYYY-MM-DDTHH:MM:SS] vcpu-0 - Migrate: Caching migration error message list:
[YYYY-MM-DDTHH:MM:SS] In(05) vcpu-0 - [msg.checkpoint.precopyfailure] Migration to host <destination host IP address> failed with error Connection reset by peer (0xbad004b).
......
[YYYY-MM-DDTHH:MM:SS]vcpu-0 - [msg.checkpoint.precopyfailure] Migration to host <destination host IP address> failed with error Connection reset by peer (0xbad004b).
Similar messages from vmkernel.log of source host :
[YYYY-MM-DDTHH:MM:SS] cpu2:######)ScsiDeviceIO: ####: Cmd(0x45ba9df2a7c8) 0x42, CmdSN 0x8b75b8 from world ####### to dev "naa.############" failed H:0x0 D:0x8 P:0x0
[YYYY-MM-DDTHH:MM:SS] cpu118:######)lpfc: lpfc_handle_status:5637: 0:(0):3271: FCP cmd x42 failed <1/4> sid x011d00, did x012700, oxid x209 iotag x52f SCSI Busy -
......
[YYYY-MM-DDTHH:MM:SS] cpu126:######)WARNING: VMotionUtil: ###: ######### S: failed to read stream keepalive: Connection reset by peer
[YYYY-MM-DDTHH:MM:SS] cpu126:######)Migrate: ###: ########## S: MigrateState: Failed
[YYYY-MM-DDTHH:MM:SS] cpu126:######)WARNING: Migrate: ###: ########### S: Failed: Connection reset by peer (0xbad004b) @0x42002ddb4d56
[YYYY-MM-DDTHH:MM:SS]cpu44:######)ScsiDeviceIO: 4115: Cmd(0x45ba9f56c7c8) 0x42, CmdSN 0x8b764f from world 29367470 to dev "naa.#############" failed H:0x0 D:0x8 P:0x0
[YYYY-MM-DDTHH:MM:SS] cpu19:#######)WARNING: SVM: ####: scsi0:0 Failed SVMFDSIoctlMoveData: Connection reset by peer
Similar messages from vmkernel.log of destination host:
[YYYY-MM-DDTHH:MM:SS] cpu21:#####)WARNING: Migrate: 7345: No migration found for world 3821944.
[YYYY-MM-DDTHH:MM:SS] cpu90:####)VMotionUtil: 7497: ############ D: Socket 0x431a876e56c0 rcvMigFree pending: 33304/33304 snd 168 rcv
[YYYY-MM-DDTHH:MM:SS] cpu14:####)ScsiDeviceIO: 4121: Cmd(0x45ba04b91a08) 0x8a, CmdSN 0x144d05 from world 3821329 to dev "naa.##############" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x29 0x3
[YYYY-MM-DDTHH:MM:SS] cpu3:####)NMP: nmp_ThrottleLogForDevice:3867: Cmd 0x2a (0x45ba04aff108, 3755606) to dev "naa.#############" on path "vmhba3:C0:T3:L2" Failed:
[YYYY-MM-DDTHH:MM:SS] cpu3:2098403)NMP: nmp_ThrottleLogForDevice:3875: H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x29 0x3. Act:NONE. cmdId.initiator=0x430dde469fc0 CmdSN 0x8000003f
Environment
VMware vSphere ESXi 7.0
VMware vSphere ESXi 8.0
Cause
The error "naa.###############" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x29 0x3 indicates that the host issued a SCSI command to the storage, but due to abnormalities on the storage side, the SCSI command was aborted. As a result, the host continuously retried the command to the storage device, eventually causing the storage device to undergo a power-on reset.
Resolution
- To check the Fabric Connectivity between host and FC Storage, including physical FC switch, FC cables, SFP.
- Engage FC Storage array vendor for support
Feedback
