vMotion fails on vCenter 6.7 Update 1 with error "The source detected destination failed to resume"
Article ID: 324322
Updated On:
Products
VMware vCenter Server
Issue/Introduction
Symptoms:
vMotion fails on vCenter 6.7 with error "The source detected destination failed to resume"
In vmware.log you will see errors similar to:
vmkernel.log:
When you check the vpxd.log, you see that vpxd is crashing with the below backtrace
vMotion fails on vCenter 6.7 with error "The source detected destination failed to resume"
In vmware.log you will see errors similar to:
2019-04-18T20:45:10.979Z| vmx| I125: MigrateDumperRead: Read 10161 bytes @ 1168756
2019-04-18T20:45:10.979Z| vmx| I125: restoring smram
2019-04-18T20:45:10.979Z| vmx| I125: DUMPER: Block item 'smram' [-1, -1] not found.
2019-04-18T20:45:10.979Z| vmx| I125: Progress 0% (none)
2019-04-18T20:45:10.979Z| vmx| I125: Progress 101% (none)
2019-04-18T20:45:10.979Z| vmx| I125: MigrateSetStateFinished: type=2 new state=12
2019-04-18T20:45:10.979Z| vmx| I125: MigrateSetState: Transitioning from state 11 to 12.
2019-04-18T20:45:10.979Z| vmx| I125: Migrate: Caching migration error message list:
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.migration.failedReceive] Failed to receive migration.
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.mrestoregroup.failed] An error occurred restoring the virtual machine state during migration.
2019-04-18T20:45:10.979Z| vmx| I125: Msg_Post: Error
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.mrestoregroup.failed] An error occurred restoring the virtual machine state during migration.
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.migration.failedReceive] Failed to receive migration
2019-04-18T20:45:10.979Z| vmx| I125: restoring smram
2019-04-18T20:45:10.979Z| vmx| I125: DUMPER: Block item 'smram' [-1, -1] not found.
2019-04-18T20:45:10.979Z| vmx| I125: Progress 0% (none)
2019-04-18T20:45:10.979Z| vmx| I125: Progress 101% (none)
2019-04-18T20:45:10.979Z| vmx| I125: MigrateSetStateFinished: type=2 new state=12
2019-04-18T20:45:10.979Z| vmx| I125: MigrateSetState: Transitioning from state 11 to 12.
2019-04-18T20:45:10.979Z| vmx| I125: Migrate: Caching migration error message list:
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.migration.failedReceive] Failed to receive migration.
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.mrestoregroup.failed] An error occurred restoring the virtual machine state during migration.
2019-04-18T20:45:10.979Z| vmx| I125: Msg_Post: Error
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.mrestoregroup.failed] An error occurred restoring the virtual machine state during migration.
2019-04-18T20:45:10.979Z| vmx| I125: [msg.checkpoint.migration.failedReceive] Failed to receive migration
vmkernel.log:
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: Migrate: 5454: 4421787300420266144 D: Migration considered a failure by the VMX. It is most likely a timeout, but check the VMX log for the true error.
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: Migrate: 273: 4421787300420266144 D: Failed: Migration determined a failure by the VMX (0xbad0092) @0x4180369fc3b3
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: VMotionUtil: 6267: 4421787300420266144 D: timed out waiting 0 ms to transmit data.
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: VMotionSend: 688: 4421787300420266144 D: (9-0x43d240001aa8) failed to receive 72/72 bytes from the remote host <172.28.2.12>: Timeout
2019-04-18T21:10:16.410Z cpu10:27294735)WARNING: VMotionUtil: 6111: 4421787301926567010 D: Bandwidth of vMotion vmknic vmk2 could not be estimated, status: Not found.
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: Migrate: 273: 4421787300420266144 D: Failed: Migration determined a failure by the VMX (0xbad0092) @0x4180369fc3b3
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: VMotionUtil: 6267: 4421787300420266144 D: timed out waiting 0 ms to transmit data.
2019-04-18T20:45:10.979Z cpu3:27293185)WARNING: VMotionSend: 688: 4421787300420266144 D: (9-0x43d240001aa8) failed to receive 72/72 bytes from the remote host <172.28.2.12>: Timeout
2019-04-18T21:10:16.410Z cpu10:27294735)WARNING: VMotionUtil: 6111: 4421787301926567010 D: Bandwidth of vMotion vmknic vmk2 could not be estimated, status: Not found.
When you check the vpxd.log, you see that vpxd is crashing with the below backtrace
2019-04-18T20:31:44.173Z error vpxd[04301] [Originator@6876 sub=VmProv opID=MigrationWizard-applyOnMultiEntity-96027-ngc:70011817-ab-01] Failed to track task vim.Task:task-39803 on host vim.HostSystem:host-103: Fault cause: vim.fault.
GenericVmConfigFault
-->
--> backtrace:
--> [backtrace begin] product: VMware VirtualCenter, version: 6.7.0, build: build-10244857, tag: vpxd, cpu: x86_64, os: linux, buildType: release
--> backtrace[00] libvmacore.so[0x002AB094]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)
--> backtrace[01] libvmacore.so[0x001B3720]: Vmacore::System::SystemFactory::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)
--> backtrace[02] libvmacore.so[0x0018BB6E]: Vmacore::Throwable::Throwable(std::string&&)
--> backtrace[03] libvim-types.so[0x00FC2B24]
--> backtrace[04] libvim-types.so[0x010F50FB]
--> backtrace[05] libvim-types.so[0x010FFE97]
--> backtrace[06] libvim-types.so[0x011011DA]: Vim::Fault::GenericVmConfigFault::ThrowInternal()
--> backtrace[07] vpxd[0x0054FC9B]
--> backtrace[08] vpxd[0x00715374]
--> backtrace[09] vpxd[0x00CC4258]
--> backtrace[10] vpxd[0x00CC4826]
--> backtrace[11] vpxd[0x00CD614A]
--> backtrace[12] vpxd[0x00C9FADF]
--> backtrace[13] vpxd[0x00CA1A17]
--> backtrace[14] vpxd[0x00CA1ED0]
--> backtrace[15] vpxd[0x0070DF76]
--> backtrace[16] vpxd[0x0070ED27]
--> backtrace[17] vpxd[0x0070CA2A]
--> backtrace[18] vpxd[0x00C62CA6]
--> backtrace[19] vpxd[0x00A83EF5]
--> backtrace[20] vpxd[0x00A83FF4]
--> backtrace[21] libvim-types.so[0x01012DBB]
--> backtrace[22] vpxd[0x0071DB2B]
--> backtrace[23] vpxd[0x0070DF76]
--> backtrace[24] vpxd[0x0070ED27]
--> backtrace[25] vpxd[0x007147CE]
--> backtrace[26] libvmacore.so[0x00231EA1]
--> backtrace[27] libvmacore.so[0x0023216A]
--> backtrace[28] libvmacore.so[0x002B081D]
--> backtrace[29] libpthread.so.0[0x000073D4]
--> backtrace[30] libc.so.6[0x000E8C9D]
--> [backtrace end]
GenericVmConfigFault
-->
--> backtrace:
--> [backtrace begin] product: VMware VirtualCenter, version: 6.7.0, build: build-10244857, tag: vpxd, cpu: x86_64, os: linux, buildType: release
--> backtrace[00] libvmacore.so[0x002AB094]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)
--> backtrace[01] libvmacore.so[0x001B3720]: Vmacore::System::SystemFactory::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)
--> backtrace[02] libvmacore.so[0x0018BB6E]: Vmacore::Throwable::Throwable(std::string&&)
--> backtrace[03] libvim-types.so[0x00FC2B24]
--> backtrace[04] libvim-types.so[0x010F50FB]
--> backtrace[05] libvim-types.so[0x010FFE97]
--> backtrace[06] libvim-types.so[0x011011DA]: Vim::Fault::GenericVmConfigFault::ThrowInternal()
--> backtrace[07] vpxd[0x0054FC9B]
--> backtrace[08] vpxd[0x00715374]
--> backtrace[09] vpxd[0x00CC4258]
--> backtrace[10] vpxd[0x00CC4826]
--> backtrace[11] vpxd[0x00CD614A]
--> backtrace[12] vpxd[0x00C9FADF]
--> backtrace[13] vpxd[0x00CA1A17]
--> backtrace[14] vpxd[0x00CA1ED0]
--> backtrace[15] vpxd[0x0070DF76]
--> backtrace[16] vpxd[0x0070ED27]
--> backtrace[17] vpxd[0x0070CA2A]
--> backtrace[18] vpxd[0x00C62CA6]
--> backtrace[19] vpxd[0x00A83EF5]
--> backtrace[20] vpxd[0x00A83FF4]
--> backtrace[21] libvim-types.so[0x01012DBB]
--> backtrace[22] vpxd[0x0071DB2B]
--> backtrace[23] vpxd[0x0070DF76]
--> backtrace[24] vpxd[0x0070ED27]
--> backtrace[25] vpxd[0x007147CE]
--> backtrace[26] libvmacore.so[0x00231EA1]
--> backtrace[27] libvmacore.so[0x0023216A]
--> backtrace[28] libvmacore.so[0x002B081D]
--> backtrace[29] libpthread.so.0[0x000073D4]
--> backtrace[30] libc.so.6[0x000E8C9D]
--> [backtrace end]
Environment
VMware vCenter Server 6.7.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server Appliance 6.7.x
Cause
The problem happens only if there is a namespaceMgr.dbFile config entry in the vmx file where the namespace db file does not exist.
Resolution
The issue has been resolved in vCenter 6.7 Update 2, you can download here
Workaround:
Current workaround is to power off the VM, re-register the VM and then do the migration
Workaround:
Current workaround is to power off the VM, re-register the VM and then do the migration
Feedback
Yes
No
Powered by
