This article provides information about overriding vMotion CPU compatibility restrictions between ESX hosts managed by VirtualCenter 2.x or vCenter Server 4.x. For information on overriding vMotion CPU compatibility restrictions between ESX hosts managed by VirtualCenter 1.x, see vMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks for VirtualCenter 1.x (1011354).
Warning: Unless indicated, VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration. Experimental support is available for some masks in ESX 3.5 and later as noted. For more information on experimental support, see http://www.vmware.com/support/policies/experimental.html.
VMware recommends using Enhanced vMotion Compatibility (EVC) to eliminate most vMotion CPU compatibility problems. For more information on EVC, see Enhanced vMotion Compatibility (EVC) processor support (1003212).
To determine if the source and target CPUs meet vMotion requirements, VirtualCenter and vCenter Server compare the target CPU to a default bit mask definition to exclude unimportant features from the comparison. As new features are introduced by processor vendors, the bit masks used by VirtualCenter and vCenter Server are updated. For example, in ESX Server 2.x, the NX (AMD) or XD (Intel) feature is not used to compute vMotion compatibility, but in ESX 3.x, it is.
To disable vMotion’s CPU-compatibility checking for a specific feature, you can modify the default bit mask. The bit mask-modification process is sometimes referred to in some VMware documents as “relaxing” a particular constraint.
A full discussion of the underlying components and how VirtualCenter Server and ESX Server host systems handle them is beyond the scope of this article. For more information, see CPU Compatibility Masks in Basic System Administration. Another useful resource is the technical paper on VMware vMotion and CPU Compatibility.
For specific information about Intel and AMD CPUs and vMotion compatibility, see:
Mask | ESX 2.x | ESX 3.x | ESX 4.x |
CMPXCHG16B (AMD) | Not supported | Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. | Experimentally supported |
FFXSR (AMD) | Not supported | Not supported | Not supported |
NX (AMD) | Supported | Supported | Supported |
RDTSCP (AMD) | Not supported | Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. | Experimentally supported |
SSE3 (AMD) | Not supported | Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. | Experimentally supported |
SSE3 (Intel) | Not supported | Not supported | Not supported |
SSE4.1 (Intel) | Not supported | Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. | Experimentally supported |
SSE4.2 (Intel) | Not supported | Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. | Experimentally supported |
XD (Intel) | Supported | Supported | Supported |
AES (Intel) | Not supported | Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. | Experimentally supported |
From here the mask can be modified appropriately for the selected virtual machine, depending on what needs to be masked. The following sections include steps on modifying the masks for several common CPU registers.
Issues with the NX/XD features on CPUs are common. Ensure that the feature is either enabled (or disabled) in the BIOS of all hosts to avoid these compatibility error messages.
Note: Not all servers have the option to disable or enable NX or XD in the BIOS.
If a CPU feature compatibility issue with the NX/XD bit is encountered, an error similar to this is generated:
The CPU of the host is incompatible with the cpu feature requirements of virtual machine; problem detected at CPUID level 0x80000001 register 'edx'.
Power off the virtual machine before modifying the NX/XD mask.
To modify the mask to enable or disable the NX/XD CPU bit:
Masks for several other features can be modified, to provide further compatibility for vMotion. Support terms for these feature masks are listed in vMotion CPU Compatibility Requirements for Intel Processors (1991). In general, VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.
Power off the virtual machine before modifying its bit mask.
To modify other CPU masks:
To modify the mask for a specific feature, enter the series of dashes and 0s as shown in this table:
Feature | Level | Row | Mask |
SSE3 | 1 | ecx | ---- ---- ---- ---- ---- ---- ---0 -0-0 |
SSSE3 | 1 | ecx | ---- ---- ---- -0-- ---- --0- ---0 -0-- |
1 | edx | ---- ---- ---- --0- ---- ---- ---- ---- | |
SSE4.1 | 1 | ecx | ---- ---- ---- 0--- ---- ---- ---- ---- |
SSE4.2 | 1 | ecx | ---- ---- 0--0 ---- ---- ---- ---- ---- |
80000001 | edx | ---- 0--- ---- ---- ---- ---- ---- ---- | |
AES | a | ecx | ---- --0- ---- ---- ---- ---- ---- --0- |
When all modifications are complete, click OK and exit the CPU Identification Mask dialog box.
Features that are specific to AMD processors are listed and can be changed on the AMD Override tab. Support terms for these feature masks are listed in vMotion CPU Compatibility Requirements for AMD Processors (1992). In general, VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.
Power off the virtual machine before modifying its bit mask.
To modify other CPU masks for AMD Processors:
To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.
Feature | Level | Row | Mask |
SSE3 | 1 | ecx | ---- ---- ---- ---- ---- ---- ---- ---0 |
RDTSCP | 80000001 | edx | ---- 0--- ---- ---- ---- ---- ---- ---- |
80000001 | ecx | ---- ---- ---- ---- ---- ---- ---- 0--- | |
CMPXCH16B | 1 | ecx | ---- ---- ---- ---- --0- ---- ---- ---- |
FFXSR | 80000001 | edx | ---- --0- ---- ---- ---- ---- ---- ---- |
When all modifications are complete click OK and exit the CPU Identification Mask dialog box.
You can combine modifications to the default bit mask to allow migration with vMotion between groups that are incompatible based on more than one CPU feature. For example, to filter-out the compatibility check for both SSE3 and SSSE3 combine the following:
---- ---- ---- ---- ---- ---- ---0 -0-0
and:
---- ---- ---- -0-- ---- --0- ---0 -0--
to yield:
---- ---- ---- -0-- ---- --0- ---0 -0-0
for the ecx register mask.
For ESX/ESXi hosts managed by VirtualCenter or vCenter Server, you can modify the bit mask globally by manually editing the VirtualCenter or vCenter Server configuration file (vpxd.cfg ). The configuration file, vpxd.cfg , contains XML tags defining various elements and settings for the VirtualCenter Server or vCenter Server system. Bit-masks can be modified by adding the appropriate XML tags to define the mask in the context of a guest operating system configuration option to the configuration file .
Changes to the global bit mask do not take effect for individual virtual machines until they have been powered off and powered on again.
Note: The vpxd.cfg file is located by default in C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter
Although a full discussion of building these elements is beyond the scope of this article, the About the Tags section provides a brief overview. See the following section for specific steps related to Editing the Configuration File and XML tags for Common Mask Patterns like, SSE3 , SSSE3 , and SSE3 and SSSE3 Combined .
Warning: VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.
The tags identify the register to which the mask applies, and for which hosts and which versions the mask applies. The tags are direct descendants of the <vpxd> tag, in the vpxd.cfg file. Tags are listed in the order in which they must be nested:
<guestOSDescriptor> — Direct descendant (child) tag of <vpxd> element. The vpxd.cfg can contain a single guestOSDescriptor element. <host-product-and-version> — Between the opening and closing <guestOSDescriptor> tags, you can nest multiple host-product-and-version-tags that specify the version and host to which the masks you define apply. The <host-product-and-version> tags can be specified generally, as in <esx-2-x-x> for an ESX Server 2.x host, <esx-3-x-x> for an ESX/ESXi 3.x host or can be specified precisely, as in <esx-3-5-0> . For example, the tag <esx-2-5-x> identifies various maintenance release levels of the ESX Server to which the subsequently nested mask applies. <virtual-machine-configuration-hardware-version> — Allows for the configuration to be restricted by virtual machine hardware version. For example, <all-versions> , <vmx-03> , or <vmx-04> . In the example below <all-versions> is specified, meaning that the mask applies to all virtual machine hardware versions. <virtual-machine guest version> — Allows for a guest version to be specified. For example, <all-guests> , <winXPPro64Guest> , <winVistaGuest> , and so on. In the example shown below, <all-guests> is specified, and therefore the mask applies to all guest versions. <cpuFeatureMask> — This tag precedes the actual mask. The mask is then defined for <default-vendor> , <amd> , or <intel> , depending on your needs. Tag elements that are used to define the mask, identify the CPU mask. The element details include vendor, CPU ID level, and the CPU register. The valid choices for these are:
CPU vendor <default-vendor>, <amd>, <intel> CPU ID level <level-0>, <level-1>, <level-80000000>, <level-80000001> CPU register <eax>, <ebx>, <ecx>, <edx>
Tags must be embedded in the order shown in the table. A CPU-vendor tag is followed by a CPU-ID-level tag, followed by a CPU-register tag. Immediately following the CPU-register tag is the sequence of 32 dashes and x-s that represent the actual 32-bit register mask appropriate for the feature being modified. The mask is then followed by the CPU-register-, CPU-ID-level-, and CPU-vendor-closing tags.
Note: A mask defined using <default-vendor> tag is used by the system only when no vendor-specific mask has been specified for the same CPU ID level.
When you have finished defining masks, close with </guestOSDescriptor> .
Below is an example of the tag order, shown in the context of the vpxd.cfg file. The <config> and <vpxd> opening and closing tags are already in the vpxd.cfg file. The <guestOSDescriptor> opening tag can be placed directly after the <vpxd> opening tag, embed the appropriate tags to suit your needs, followed by the closing </guestOSDescriptor> tag.
<config>
<vpxd>
<guestOSDescriptor>
<esx-2-x-x>
<all-versions>
<all-guests>
<cpuFeatureMask>
[Elements and mask definition go in here. Common Mask Patterns can be copy-and-pasted from the below example patterns.]
</cpuFeatureMask>
</all-guests>
</all-versions>
</esx-2-x-x>
</guestOSDescriptor>
...
</vpxd>
</config>
In this example, any mask embedded in the [Elements and mask...] placeholder section applies to all ESX Server 2.x hosts being managed by the VirtualCenter Server, and to all guest operating systems.
To edit the VirtualCenter Configuration File:
<cpuFeatureMask>
<default-vendor>
<level-1>
<ecx>----:----:----:----:----:----:---x:-x-x</ecx>
</level-1>
</default-vendor>
<amd>
<level-1>
<ecx>----:----:----:----:----:----:----:---x</ecx>
</level-1>
</amd>
</cpuFeatureMask>
<cpuFeatureMask>
<default-vendor>
<level-1>
<ecx>----:----:----:-x--:----:--x-:---x:-x--</ecx>
<edx>----:----:----:--x-:----:----:----:----</edx>
</level-1>
</default-vendor>
</cpuFeatureMask>
<cpuFeatureMask>
<default-vendor>
<level-1>
<ecx>----:----:----:-x--:----:--x-:---x:-x-x</ecx>
<edx>----:----:----:--x-:----:----:----:----</edx>
</level-1>
</default-vendor>
<amd>
<level-1>
<ecx>----:----:----:----:----:----:----:---x</ecx>
</level-1>
</amd>
</cpuFeatureMask>
Revert to the default vMotion compatibility constraints by reverting any changes made to the default mask. This can be accomplished by following the appropriate steps for the type of masking that was applied: