Using Storage vMotion to migrate a virtual machine with many disks fails with timeout
search cancel

Using Storage vMotion to migrate a virtual machine with many disks fails with timeout

book

Article ID: 307365

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:
You may experience these symptoms:
  • Storage vMotion fails.
  • The Storage vMotion operation fails with a timeout between 5-10% or 90-95% complete.
  • On ESXi  you may see the errors similar to the below mentioned snippets:

    In hostd.log
    [7196 foundryVM.c:10177]: Error VIX_E_INVALID_ARG in VixVM_CancelOps(): One of the parameters was invalid 'vm:/vmfs/volumes/4e417019-########-####-##########0a/Mail02/Mail02.vmx' opID=9BED9F06-000002BE-9d] Failed to unset VM medatadata: FileIO error: Could not find file : /vmfs/volumes/4e417019-########-####-##########0a/Mail02/Mail02-aux.xml.tmp.

    In vmware.log

    vmkernel: 114:03:25:51.489 cpu0:4100)WARNING: FSR: 690: 1313159068180024 S: Maximum switchover time (100 seconds) reached. Failing migration; VM should resume on source.
    vmkernel: 114:03:25:51.489 cpu2:10561)WARNING: FSR: 3281: 1313159068180024 D: The migration exceeded the maximum switchover time of 100 second(s). ESX has preemptively failed the migration to allow the VM to continue running on the source host.
    vmkernel: 114:03:25:51.489 cpu2:10561)WARNING: Migrate: 296: 1313159068180024 D: Failed: Maximum switchover time for migration exceeded(0xbad0109) @0x41800f61cee2

  • vCenter Server logs contain entries similar to:

    [yyyy-mm-dd hh:mm:ss.nnn tttt error 'App'] [MIGRATE] (migrateidentifier) vMotion failed: vmodl.fault.SystemError
    [yyyy-mm-dd hh:mm:ss.nnn tttt verbose 'App'] [VpxVmomi] Throw vmodl.fault.SystemError with:
    (vmodl.fault.SystemError) {
    dynamicType = <unset>,
    reason = "Source detected that destination failed to resume.",
    msg = "A general system error occurred: Source detected that destination failed to resume.


Environment

VMware vSphere ESXi 5.1
VMware vSphere ESXi 7.0.0
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7
VMware vSphere ESXi 5.5
VMware ESXi 4.1.x Embedded
VMware ESX 4.1.x
VMware vSphere ESXi 5.0
VMware ESX 4.0.x
VMware ESXi 4.0.x Installable
VMware ESXi 4.0.x Embedded
VMware ESXi 4.1.x Installable

Resolution

Note: A virtual machine with many virtual disks might be unable to complete a migration with Storage vMotion. The Storage vMotion process requires time to open, close, and process disks during the final copy phase. Storage vMotion migration of virtual machines with many disks might timeout because of this per-disk overhead.

This timeout occurs when the maximum amount of time for switchover to the destination is exceeded. This may occur if there are a large number of provisioning, migration, or power operations occurring on the same datastore as the Storage vMotion. The virtual machine's disk files are reopened during this time, so disk performance issues or large numbers of disks may lead to timeouts.

The default timeout is 100 seconds, and can be modified by changing the fsr.maxSwitchoverSeconds option in the virtual machine configuration to a larger value.

Note: Ensure this change is performed when the virtual machine is powered down.

Process to modify fsr.maxSwitchoverSeconds using vCenter web Client/ESXi Host Client:

Follow the below steps to modify the fsr.maxSwitchoverSeconds option using the vCenter web Client or ESXi Host Client:

  1. Login to vCenter web Client or ESXi Host Client

  2. Locate the virtual machine in the Inventory.

  3. Power off the virtual machine.

  4. Right-click the virtual machine and click Edit Settings.

  5. Click the VM Options tab.

  6. Select the Advanced.

  7. Click the Edit Configuration button.
    Note: The Configuration Parameters button is disabled when the virtual machine is powered on. 

  8. From the Configuration Parameters window, click Add Configuration Params.

  9. In the Name field, enter the parameter name:

            fsr.maxSwitchoverSeconds

  1. In the Value field, enter the new timeout value in seconds (for example: 200)
  2. Click OK .
  3. Power on the virtual machine.

Process to modify fsr.maxSwitchoverSeconds option by editing .vmx file manually:

To modify the fsr.maxSwitchoverSeconds option by editing the .vmx file manually follow the steps below:

The virtual machine's .vmx configuration file can be manually edited to add or modify the option. Add the option fsr.maxSwitchoverSeconds = " <value></value><new value>" on its own line.

For more information, see Tips for editing a .vmx file.


Note: To edit a virtual machines configuration file, you need to power off the virtual machine, remove it from Inventory, make the changes to the vmx file, add the virtual machine back to inventory, and then power on the virtual machine again OR follow the article Reloading a vmx file without removing the virtual machine from inventory

During Storage vMotion, this issue can occur if the host is unable to copy the swap file activity of the virtual machine within a default time of 100 seconds from the source datastore to the destination datastore. It can also occur if the source datastore or the destination datastore are having any performance related issues because of heavy I/O activity.

For more information, see vMotion or Storage vMotion of a VM fails with the error: The migration has exceeded the maximum switchover time of 100 second(s)

Additional Information

Powered by Wolken Software