vMotion fails to migrate between EVC clusters of ESXi host with the same configuration

Products

• VMware vCenter Server

Issue/Introduction

This article provides information to resolve issues with vMotion between ESXi hosts with the same configuration.

Symptoms:

• Unable to live vMotion a virtual machine
• During a migration an error similar to below is shown:

com.vmware.vim.vmfeature.cpuid.stibp
com.vmware.vim.vmfeature.cpuid.ibrs
com.vmware.vim.vmfeature.cpuid.ibpb

• Cold vMotion a virtual machine is successful. 

Note: To resolve CPU incompatibilities, use a cluster with Enhanced vMotion Compatibility (EVC) enabled. See KB article 1003212

Cause

This is a known behavior because of Spectre and Meltdown vulnerability patching.

This issue occurs because the source host has been patched for the Spectre/Meltdown mitigation and the destination hosts attempted have not been patched.

For information on the Spectre and Meltdown vulnerabilities, see:

• VMware Response to Speculative Execution security issues, CVE-2017-5753, CVE-2017-5715, CVE-2017-5754, and CVE-2018-3693 (aka Spectre and Meltdown)
• VMware Response to Speculative Execution security issues, CVE-2018-3639 and CVE-2018-3640

Resolution

This issue is resolve in vSphere 6.5 Update 2, available at Broadcom Support Portal .
 
Note: All ESXi hosts must be updated to at minimum 6.5 Update 2 to resolve this issue.

Workaround:
To workaround this issue, add these options to the virtual machines configure (.vmx) file:
featMask.vm.cpuid.stibp = "Max:0"
featMask.vm.cpuid.ibrs = "Max:0"
featMask.vm.cpuid.ibpb = "Max:0"

1. Using the vSphere Client, connect to the vCenter Server and locate the affected virtual machine.
2. Power off the virtual machine.
   
   
   
3. Right-click the virtual machine and click Edit Settings > VM Options > Advanced > Configuration Parameters - Edit configuration.
   
   
   
   
4. Add the below parameters:

featMask.vm.cpuid.stibp = "Max:0"
featMask.vm.cpuid.ibrs = "Max:0"
featMask.vm.cpuid.ibpb = "Max:0"

5. Click OK and OK, power on the virtual machine and migrate.