Storage vMotion tasks are failing for powered on VMs with multi-writer disks
search cancel

Storage vMotion tasks are failing for powered on VMs with multi-writer disks

book

Article ID: 318799

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • Attempts to migrate a virtual machine between datastores are failing.
  • vSphere Client is reporting an error 195887107:

Failed waiting for data. Error 195887107. Not found. Failed to start SvMotion. Error during disk setup. A fatal internal error occurred. See the virtual machine's log for more details

  • Alternatively an Error 195887250 might be reported:

 Failed waiting for data. Error 195887250. Migration determined a failure by the VMX.

  • The vmware.log file of the affected virtual machine, located in the virtual machine folder /vmfs/volumes/<datastore>/<vm_name>/ contains entries similar to:
    Wa(03) worker-28025399 - Mirror: (digest)scsi#:#: SVMotionDiskSetup: You may not migrate a multi-writer disk
Wa(03) worker-28025399 - SVMotionPrepareForCopyThread: Failed to start SvMotion. Error during disk setup.
In(05) worker-28025399 - SVMotion: FailureCleanup thread completes.
In(05) vmx - MigrateWriteHostLog: Writing to log file took 3181 us.
In(05) vmx - MigrateSetStateFinished: type=1 new state=6
In(05) vmx - MigrateSetState: Transitioning from state 2 to 6.
No(00) vmx - ConfigDB: Setting config.readOnly = "FALSE"
In(05) vmx - Migrate_SetFailureMsgList: switching to new log file.
In(05) vmx - Migrate_SetFailureMsgList: Now in new log file.
In(05) vmx - Migrate: Caching migration error message list:
In(05) vmx - [msg.svmotion.fail.internal] A fatal internal error occurred. See the virtual machine's log for more details.
In(05) vmx - [msg.svmotion.mirror.disk.setup.error] Failed to start SvMotion. Error during disk setup.
In(05) vmx - Migrate: cleaning up migration state.
In(05) vmx - SVMotion: Enter Phase 12
In(05) vmx - SVMotion_Cleanup: Scheduling cleanup thread.
  • /var/run/log/vmkernel.logon of the ESXi host holds messages like these:
    Hbr: 3395: Migration start received (worldID=####) (migrateType=2) (event=0) (isSource=0) (sharedConfig=0)
cpu#:###)Migrate: vm####: 4049: Setting FSR info: Source ts = #########, src ip = <###.###.###.###> dest ip = <###.###.###.###> Dest wid = #### using UNSHARED swap
cpu#:###)Hbr: 3395: Migration start received (worldID=76253) (migrateType=2) (event=0) (isSource=1) (sharedConfig=0)
WARNING: Migrate: 6279: ######## S: Migration considered a failure by the VMX.  It is most likely a timeout, but check the VMX log for the true error.
cpu#:###)WARNING: Migrate: 273: ######## S: Failed: Migration determined a failure by the VMX (0xbad0092) @0x41801671xxxx
Hbr: 3489: Migration end received (worldID=########) (migrateType=2) (event=1) (isSource=1) (sharedConfig=0)
cpu#:###)WARNING: FSR: 355: ######## D: Could not find migrate info for remote world 76253!
cpu#:###)WARNING: FSR: 4255: ######## D: Failed to find remote FSR.  Did the migration fail?
cpu#:###)Config: 706: "SIOControlFlag2" = 1, Old Value: 2, (Status: 0x0)
cpu#:###)WARNING: Migrate: 6279: ###### D: Migration considered a failure by the VMX.  It is most likely a timeout, but check the VMX log for the true error.
cpu#:###)WARNING: Migrate: 273: ######## D: Failed: Migration determined a failure by the VMX (0x#####) @0x#######



Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

  • VMware vSphere ESXi 7.0.x
  • VMware vSphere ESXi 8.0.x

Cause

This issue occurs when one or more of the virtual machines disks are configured as multi-writer disk.
 
Multi-writer disks are usually shared between multiple virtual machines, which are concurrently accessing these disks. To prevent disk access failures for the other VMs accessing such disks, Storage vMotion is not supported for any disks attached to a virtual machine which has a multi-writer disk, no matter if the specific disk is shared or not.
 

Resolution

Storage vMotion is not supported for multi-writer (shared) disks.

To migrate non-multi-writer disks that are attached to a virtual machine which has both shared and non-shared disks attached, the following workaround can be used:

  1. Power the virtual machine off.
  2. Perform the Storage vMotion.
  3. Power the VM back on again once the storage vMotion task has been completed.

Attention: Do not attempt to migrate the actual multi-writer disk(s) between datastores. Doing so will invalidate the link(s) for these disks in the other VMs configuration and is not supported.

Additional Information

Review any backup software tools messages about disk space issues. Snapshots of the virtual machine will impact storage vMotions if space on the  LUN is less than 1.5 times the total of the VM's snapshots sizes. Confirm with your company storage team the LUN the virtual machine resides on has enough over head space to consolidate data and migrate. If not expand the current LUN where the virtual machine resides and consolidate the data. If the LUN can not expand space a clone of the Virtual Machines disk to another datastore is an option Cloning and converting virtual machine disks with vmkfstools (343140)


The man page on the ESXI host use the command: vmkfstools -h

example syntax:  
vmkfstools -i "/vmfs/volumes/currentDatastore/vm_name/vm_name.vmdk" "/vmfs/volumes/newDatastore/vm_name/vm_name.vmdk" -d thin -a buslogic

 

 

Powered by Wolken Software