Virtual machines with an invalid coresPerSocket configuration cannot be vMotioned to an 7.0.3 host from some patch levels of ESXi

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

When attempting to vMotion certain VMs to an 7.0 U3 ESXi host, it might fail with the following error message in the vSphere Client:

“Failed to receive migration. The source detected that the destination failed to resume."

This happens when the following conditions are fulfilled:

• vCPU HotAdd enabled on the virtual machine
• cpuid.coresPerSocket has a larger value than the assigned number of vCPUs (invalid)

Environment

VMware vSphere ESXi 7.0.3

Cause

ESXi versions before 7.0.3 would internally disable vCPU HotAdd when an invalid coresPerSocket value was configured, this is no longer the case so a VM’s hotplug capability differs across source and destination. This difference in capability causes the virtual SMRAM device to use different modes. When vCPU HotAdd is disabled, it uses the memory backing mode, which does not need to checkpoint during the hot migration. When vCPU HotAdd is enabled, it uses the device backing mode, which requires the device to be checkpointed and restored across the vMotion. When VMs with those specific configurations are hot migrated to an ESXi 7.0.3 and restored on the destination host, they expect to retrieve a checkpointed SMRAM device but none is present, this causes the migration to fail.

Resolution

This issue is resolved in vSphere ESXi 7.0 U3f (build number 20036589).

Workaround:
To work around this issue, you may either:

• Power off the virtual machine and correct the invalid coresPerSocket setting
• Power off the virtual machine and cold migrate it do the destination
• Configure the virtual machine advanced option, "chipset.forceLegacySmram = TRUE" via the vSphere API before the vMotion, note that this will prevent the VM from hot migrating back to the non 7.0 U3 source.
  • Example snippet in PowerCLI:

Get-VM <-name_your_vm_or_other_conditons> | New-AdvancedSetting -Name chipset.forceLegacySmram -Value TRUE -Confirm:$false

Please note that the first and second methods are preferable over the third to avoid the complexity of setting further advanced settings, even if just temporarily.

Additional Information

Data Object - OptionValue