Storage vMotion fails with Error 195887107

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information to resolve an issue when a storage vMotion task failure occurs when the powered on virtual machine is using disks with the multi-writer attribute.

Symptoms:
When running a storage vMotion task on a powered on virtual machine, you experience these symptoms:

The storage vMotion task fails.
You see the error similar to:

Failed waiting for data. Error 195887107. Not found. Failed to start SvMotion. Error during disk setup. A fatal internal error occurred. See the virtual machine's log for more details.
In the vmware.log file of the affected virtual machine, you see entries similar to:

Worker#2| W115: Mirror: scsi1:0: SVMotionDiskSetup: You may not migrate a multi-writer disk
In the /var/log/vmkernel.log file of the ESXi host, you see entries similar to:

Hbr: 3395: Migration start received (worldID=134xxx5) (migrateType=2) (event=0) (isSource=0) (sharedConfig=0)
2 cpu21:76251)Migrate: vm 76253: 4049: Setting FSR info: Source ts = 881967184xxxxxxxx966, src ip = <127.0.0.x> dest ip = <127.0.0.x> Dest wid = 1347xxx using UNSHARED swap
cpu21:76251)Hbr: 3395: Migration start received (worldID=76253) (migrateType=2) (event=0) (isSource=1) (sharedConfig=0)
WARNING: Migrate: 6279: 881967184xxxxxxxx966 S: Migration considered a failure by the VMX. It is most likely a timeout, but check the VMX log for the true error.
cpu33:76251)WARNING: Migrate: 273: 881967184xxxxxxxx966 S: Failed: Migration determined a failure by the VMX (0xbad0092) @0x41801671xxxx
Hbr: 3489: Migration end received (worldID=762xx3) (migrateType=2) (event=1) (isSource=1) (sharedConfig=0)
cpu0:331719)WARNING: FSR: 355: 88196718401xxxxx966 D: Could not find migrate info for remote world 76253!
cpu0:331719)WARNING: FSR: 4255: 88196718401xxxxx966 D: Failed to find remote FSR. Did the migration fail?
cpu26:68707)Config: 706: "SIOControlFlag2" = 1, Old Value: 2, (Status: 0x0)
cpu4:331719)WARNING: Migrate: 6279: 881967184010xxxxx66 D: Migration considered a failure by the VMX. It is most likely a timeout, but check the VMX log for the true error.
cpu4:331719)WARNING: Migrate: 273: 881967184010xxxxx66 D: Failed: Migration determined a failure by the VMX (0xbad0092) @0x41801671xxxx
Virtual Machin logs.
Wa(03) worker-28025399 - Mirror: (digest)scsi1:3: SVMotionDiskSetup: You may not migrate a multi-writer disk
Wa(03) worker-28025399 - SVMotionPrepareForCopyThread: Failed to start SvMotion. Error during disk setup.
In(05) worker-28025399 - SVMotion: FailureCleanup thread completes.
In(05) vmx - MigrateWriteHostLog: Writing to log file took 3181 us.
In(05) vmx - MigrateSetStateFinished: type=1 new state=6
In(05) vmx - MigrateSetState: Transitioning from state 2 to 6.
No(00) vmx - ConfigDB: Setting config.readOnly = "FALSE"
In(05) vmx - Migrate_SetFailureMsgList: switching to new log file.
In(05) vmx - Migrate_SetFailureMsgList: Now in new log file.
In(05) vmx - Migrate: Caching migration error message list:
In(05) vmx - [msg.svmotion.fail.internal] A fatal internal error occurred. See the virtual machine's log for more details.
In(05) vmx - [msg.svmotion.mirror.disk.setup.error] Failed to start SvMotion. Error during disk setup.
In(05) vmx - Migrate: cleaning up migration state.
In(05) vmx - SVMotion: Enter Phase 12
In(05) vmx - SVMotion_Cleanup: Scheduling cleanup thread.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

Cause

This issue occurs due to the virtual machine using disk(s) configured with the multi-writer attribute.

As the virtual machine is using disk(s) with the multi-writer attribute, neither shared nor non-shared disks of the VM can be migrated using storage vMotion as the virtual machine stun is required to initiate the storage migration.

Resolution

This is an expected behavior when running a storage vMotion task on powered on virtual machines that are using disks with the multi-writer attribute.
The multi-writer option allows VMFS-based disks to be shared by multiple virtual machines. For more information, see https://knowledge.broadcom.com/external/article?articleNumber=313527

Workaround:

To work around this issue:

Power off the affected virtual machine.
Perform the storage vMotion.
Power on the virtual machine once the storage vMotion task is completed.

To address Storage vMotion errors encountered with Virtual Machines (VMs) that are not configured with multi-writer disks, please follow the sequential procedure outlined below:

Procedure:

Perform Compute vMotion (Host Migration):
- Initiate a compute vMotion to migrate the affected Virtual Machine to a different ESXi host within the same cluster.
- Ensure this operation completes successfully before proceeding to the next step.
Perform Storage vMotion (Datastore Migration):
- Once the VM is successfully running on the new ESXi host, initiate a Storage vMotion to migrate its virtual disks to the intended target datastore.

Additional Information

Review any backup software tools messages about disk space issues. Snapshots of the virtual machine will impact storage vMotions if space on the LUN is less than 1.5 times the total of the VM's snapshots sizes. Confirm with your company storage team the LUN the virtual machine resides on has enough over head space to consolidate data and migrate. If not expand the current LUN where the virtual machine resides and consolidate the data. If the LUN can not expand space a clone of the Virtual Machines disk to another datastore is an option Cloning and converting virtual machine disks with vmkfstools (343140)

The man page on the ESXI host use the command: vmkfstools -h

example syntax:
vmkfstools -i "/vmfs/volumes/currentDatastore/vm_name/vm_name.vmdk" "/vmfs/volumes/newDatastore/vm_name/vm_name.vmdk" -d thin -a buslogic