vMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch

search cancel

vMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks

book

Article ID: 341294

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

This article provides information about overriding vMotion CPU compatibility restrictions between ESX hosts managed by VirtualCenter 2.x or vCenter Server 4.x. For information on overriding vMotion CPU compatibility restrictions between ESX hosts managed by VirtualCenter 1.x, see vMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks for VirtualCenter 1.x (1011354).

Warning: Unless indicated, VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration. Experimental support is available for some masks in ESX 3.5 and later as noted. For more information on experimental support, see http://www.vmware.com/support/policies/experimental.html.

VMware recommends using Enhanced vMotion Compatibility (EVC) to eliminate most vMotion CPU compatibility problems. For more information on EVC, see Enhanced vMotion Compatibility (EVC) processor support (1003212).

Environment

VMware vSphere ESXi 6.0
VMware vCenter Server 5.5.x
VMware vSphere ESXi 6.7
VMware vCenter Server 5.0.x
VMware ESXi 4.1.x Installable
VMware vCenter Server 7.0.x
VMware vSphere ESXi 5.5
VMware vCenter Server 6.0.x
VMware ESXi 4.0.x Embedded
VMware ESXi 3.5.x Installable
VMware ESXi 4.0.x Installable
VMware vSphere ESXi 5.1
VMware vSphere ESXi 7.0.0
VMware vSphere ESXi 5.0
VMware vCenter Server 4.1.x
VMware vCenter Server 4.0.x
VMware vCenter Server 6.7.x
VMware ESXi 3.5.x Embedded
VMware vCenter Server 5.1.x
VMware ESXi 4.1.x Embedded

Resolution

If you have recent Intel 45nm Core 2 CPUs and were directed to this article by a message in VirtualCenter or vCenter Server, refer to vMotion errors between different Intel 45nm Core 2 revisions (1008315).

In ESX/ESXi 3.5 Update 2 and later, VMware recommends using Enhanced vMotion Compatibility (EVC) to eliminate most vMotion CPU compatibility problems. For more information on EVC, see Enhanced vMotion Compatibility (EVC) processor support (1003212).

By default, VirtualCenter and vCenter Server allow migrations with vMotion only between compatible source and destination CPUs. This article provides information about when it is appropriate to override CPU-compatibility constraints, how to override them, and provides links to additional information and resources. Topics include:

Background
- Scope of Modifications
- CPU Identification Tools
Per-virtual machine masking with ESX/ESXi and later
Global Masking with ESX Server 2.x and later

Reverting to the Default CPU Compatibility Bit Mask

Background

For vMotion purposes, compatible CPUs means that source and target CPUs must have the same manufacturer (AMD or Intel) and be members of the same basic processor family (for example, Pentium 4 or Core), and have a common set of features implemented.

To determine if the source and target CPUs meet vMotion requirements, VirtualCenter and vCenter Server compare the target CPU to a default bit mask definition to exclude unimportant features from the comparison. As new features are introduced by processor vendors, the bit masks used by VirtualCenter and vCenter Server are updated. For example, in ESX Server 2.x, the NX (AMD) or XD (Intel) feature is not used to compute vMotion compatibility, but in ESX 3.x, it is.

To disable vMotion’s CPU-compatibility checking for a specific feature, you can modify the default bit mask. The bit mask-modification process is sometimes referred to in some VMware documents as “relaxing” a particular constraint.

A full discussion of the underlying components and how VirtualCenter Server and ESX Server host systems handle them is beyond the scope of this article. For more information, see CPU Compatibility Masks in Basic System Administration. Another useful resource is the technical paper on VMware vMotion and CPU Compatibility.

For specific information about Intel and AMD CPUs and vMotion compatibility, see:

For simplicity, modifications to the default bit mask for CPU compatibility for vMotion are also referred to simply as masks, typically identified by the CPU feature—for example, the SSE3 mask in this and related KB articles 1991 and 1992.

Many masks are not supported. Experimental support is available for some masks in ESX 3.5 and later. Support terms for masks are summarized in the following table.

Mask	ESX 2.x	ESX 3.x	ESX 4.x
CMPXCHG16B (AMD)	Not supported	Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later.	Experimentally supported
FFXSR (AMD)	Not supported	Not supported	Not supported
NX (AMD)	Supported	Supported	Supported
RDTSCP (AMD)	Not supported	Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later.	Experimentally supported
SSE3 (AMD)	Not supported	Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later.	Experimentally supported
SSE3 (Intel)	Not supported	Not supported	Not supported
SSE4.1 (Intel)	Not supported	Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later.	Experimentally supported
SSE4.2 (Intel)	Not supported	Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later.	Experimentally supported
XD (Intel)	Supported	Supported	Supported
AES (Intel)	Not supported	Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later.	Experimentally supported

Scope of Modifications

There are two ways to modify the default CPU bit masks, depending on the ESX/ESXi version and individual needs.

With any ESX/ESXi or ESX Server version: You can modify the default bit mask at the VirtualCenter Server or vCenter Server level by processor manufacturer, by version number, and by other criteria. This method is not supported, and applications attempting to use the masked features might crash during migration with vMotion.
ESX/ESXi 3.x and later: You can modify the bit masks on a per-virtual machine basis using the VI Client or vSphere Client. You must power off the virtual machine before making these modifications due to the risk of application or guest operating system instability. Support terms for this method are listed in vMotion CPU Compatibility Requirements for Intel Processors (1991)(for Intel processors) and vMotion CPU Compatibility Requirements for AMD Processors (1992)(for AMD processors).

CPU Identification Tools

To obtain more information about a host system's CPU, use the CPU Identification Utility. VMware provides this as an ISO image file that can be uncompressed and used to create a bootable CD-ROM that provides CPU information about a host, even before an operating system or ESX is installed. The latest version of this tool can be found on the VMware downloads page at http://vmware.com/download/shared_utilities.html.

Per-virtual machine masking with ESX/ESXi 3.x and later

In Virtual Center 2.x and vCenter Server 4.x, you can modify the default bit mask for individual virtual machines using the VI Client or vSphere Client. Power off the virtual machine before modifying its bit mask.

To modify the default bit mask for a virtual machine:

Launch VI Client or vSphere Client and connect to the VirtualCenter Server or vCenter Server as an administrator.
Select the virtual machine that you want to migrate in the inventory.
Click Edit Settings on the Summary tab. The Properties window for the virtual machine appears.
Click the Options tab in the Properties window.
Select the CPUID Mask option to display the CPU Identification Mask information and settings.
Click Advanced to display several settings-related boxes, including CPU Identification Mask, in the right-hand pane.

From here the mask can be modified appropriately for the selected virtual machine, depending on what needs to be masked. The following sections include steps on modifying the masks for several common CPU registers.

Modifying the default NX/XD mask

Issues with the NX/XD features on CPUs are common. Ensure that the feature is either enabled (or disabled) in the BIOS of all hosts to avoid these compatibility error messages.

Note: Not all servers have the option to disable or enable NX or XD in the BIOS.

If a CPU feature compatibility issue with the NX/XD bit is encountered, an error similar to this is generated:

The CPU of the host is incompatible with the cpu feature requirements of virtual machine; problem detected at CPUID level 0x80000001 register 'edx'.

Power off the virtual machine before modifying the NX/XD mask.

To modify the mask to enable or disable the NX/XD CPU bit:

Navigate to the CPUID Mask option on the virtual machine Options tab (see steps above, if necessary).
Select Hide the NX flag from guest to disable the CPUcompatibility check for the selected virtual machine,or Expose the NX flag to guest to enable this CPU compatibility check for the selected virtual machine.
Click OK to save the change.

Modifying the default mask for Intel CPUs

Masks for several other features can be modified, to provide further compatibility for vMotion. Support terms for these feature masks are listed in vMotion CPU Compatibility Requirements for Intel Processors (1991). In general, VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.

Power off the virtual machine before modifying its bit mask.

To modify other CPU masks:

Navigate to the virtual machine Options tab (see steps above, if necessary).
Click Advanced to open the CPU Identification Mask properties dialog box.

Note: The CPU Identification Mask dialog box has two tabs—Virtual Machine Default, and AMD Override. Most modifications for Intel CPU features are made on the Virtual Machine Default tab. Modifications for AMD CPU features are made on the AMD Overrides tab, and are discussed in the next section of this article.
Click the Virtual Machine Default tab to activate the dialog box, if necessary.

To modify the mask for a specific feature, enter the series of dashes and 0s as shown in this table:

Feature	Level	Row	Mask
SSE3	1	ecx	---- ---- ---- ---- ---- ---- ---0 -0-0
SSSE3	1	ecx	`---- ---- ---- -0-- ---- --0- ---0 -0--`
SSSE3	1	edx	`---- ---- ---- --0- ---- ---- ---- ----`
SSE4.1	1	ecx	`---- ---- ---- 0--- ---- ---- ---- ----`
SSE4.2	1	ecx	---- ---- 0--0 ---- ---- ---- ---- ----
SSE4.2	80000001	edx	---- 0--- ---- ---- ---- ---- ---- ----
AES	a	ecx	---- --0- ---- ---- ---- ---- ---- --0-

When all modifications are complete, click OK and exit the CPU Identification Mask dialog box.

Modifying masks for AMD CPUs

Features that are specific to AMD processors are listed and can be changed on the AMD Override tab. Support terms for these feature masks are listed in vMotion CPU Compatibility Requirements for AMD Processors (1992). In general, VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.

Power off the virtual machine before modifying its bit mask.

To modify other CPU masks for AMD Processors:

Navigate to the virtual machine Options tab (see steps above, if necessary).
Click Advanced to open the CPU Identification Mask properties dialog box.
Click the AMD Override tab to activate the dialog box. The AMD Override page displays.

To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.

Feature	Level	Row	Mask
SSE3	1	ecx	---- ---- ---- ---- ---- ---- ---- ---0
RDTSCP	80000001	edx	`---- 0--- ---- ---- ---- ---- ---- ----`
RDTSCP	80000001	ecx	---- ---- ---- ---- ---- ---- ---- 0---
CMPXCH16B	1	ecx	---- ---- ---- ---- --0- ---- ---- ----
FFXSR	80000001	edx	---- --0- ---- ---- ---- ---- ---- ----

When all modifications are complete click OK and exit the CPU Identification Mask dialog box.

Combining mask modifications

You can combine modifications to the default bit mask to allow migration with vMotion between groups that are incompatible based on more than one CPU feature. For example, to filter-out the compatibility check for both SSE3 and SSSE3 combine the following:

---- ---- ---- ---- ---- ---- ---0 -0-0

and:

---- ---- ---- -0-- ---- --0- ---0 -0--

to yield:

---- ---- ---- -0-- ---- --0- ---0 -0-0

for the ecx register mask.

Global masking with ESX Server 2.x and later

For ESX/ESXi hosts managed by VirtualCenter or vCenter Server, you can modify the bit mask globally by manually editing the VirtualCenter or vCenter Server configuration file (vpxd.cfg ). The configuration file, vpxd.cfg , contains XML tags defining various elements and settings for the VirtualCenter Server or vCenter Server system. Bit-masks can be modified by adding the appropriate XML tags to define the mask in the context of a guest operating system configuration option to the configuration file .

Changes to the global bit mask do not take effect for individual virtual machines until they have been powered off and powered on again.

Note: The vpxd.cfg file is located by default in C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter

Although a full discussion of building these elements is beyond the scope of this article, the About the Tags section provides a brief overview. See the following section for specific steps related to Editing the Configuration File and XML tags for Common Mask Patterns like, SSE3 , SSSE3 , and SSE3 and SSSE3 Combined .

Warning: VMware neither supports nor recommends modifying the vMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.

About the tags

The tags identify the register to which the mask applies, and for which hosts and which versions the mask applies. The tags are direct descendants of the <vpxd> tag, in the vpxd.cfg file. Tags are listed in the order in which they must be nested:

<guestOSDescriptor> — Direct descendant (child) tag of <vpxd> element. The vpxd.cfg can contain a single guestOSDescriptor element.

<host-product-and-version> — Between the opening and closing <guestOSDescriptor> tags, you can nest multiple host-product-and-version-tags that specify the version and host to which the masks you define apply. The <host-product-and-version> tags can be specified generally, as in <esx-2-x-x> for an ESX Server 2.x host, <esx-3-x-x> for an ESX/ESXi 3.x host or can be specified precisely, as in <esx-3-5-0> . For example, the tag <esx-2-5-x> identifies various maintenance release levels of the ESX Server to which the subsequently nested mask applies.

<virtual-machine-configuration-hardware-version> — Allows for the configuration to be restricted by virtual machine hardware version. For example, <all-versions> , <vmx-03> , or <vmx-04> . In the example below <all-versions> is specified, meaning that the mask applies to all virtual machine hardware versions.

<virtual-machine guest version> — Allows for a guest version to be specified. For example, <all-guests> , <winXPPro64Guest> , <winVistaGuest> , and so on. In the example shown below, <all-guests> is specified, and therefore the mask applies to all guest versions.

<cpuFeatureMask> — This tag precedes the actual mask. The mask is then defined for <default-vendor> , <amd> , or <intel> , depending on your needs. Tag elements that are used to define the mask, identify the CPU mask. The element details include vendor, CPU ID level, and the CPU register. The valid choices for these are:

CPU vendor

<default-vendor>, <amd>, <intel>

CPU ID level

<level-0>, <level-1>, <level-80000000>, <level-80000001>

CPU register

<eax>, <ebx>, <ecx>, <edx>

Tags must be embedded in the order shown in the table. A CPU-vendor tag is followed by a CPU-ID-level tag, followed by a CPU-register tag. Immediately following the CPU-register tag is the sequence of 32 dashes and x-s that represent the actual 32-bit register mask appropriate for the feature being modified. The mask is then followed by the CPU-register-, CPU-ID-level-, and CPU-vendor-closing tags.

Note: A mask defined using <default-vendor> tag is used by the system only when no vendor-specific mask has been specified for the same CPU ID level.

When you have finished defining masks, close with </guestOSDescriptor> .

Below is an example of the tag order, shown in the context of the vpxd.cfg file. The <config> and <vpxd> opening and closing tags are already in the vpxd.cfg file. The <guestOSDescriptor> opening tag can be placed directly after the <vpxd> opening tag, embed the appropriate tags to suit your needs, followed by the closing </guestOSDescriptor> tag.

<config>
<vpxd>
<guestOSDescriptor>
<esx-2-x-x>
<all-versions>
<all-guests>
<cpuFeatureMask>
[Elements and mask definition go in here. Common Mask Patterns can be copy-and-pasted from the below example patterns.]
</cpuFeatureMask>
</all-guests>
</all-versions>
</esx-2-x-x>
</guestOSDescriptor>
...

</vpxd>

</config>

In this example, any mask embedded in the [Elements and mask...] placeholder section applies to all ESX Server 2.x hosts being managed by the VirtualCenter Server, and to all guest operating systems.

Editing the VirtualCenter configuration file

To edit the VirtualCenter Configuration File:

Navigate to the location of the vpxd.cfg file. By default, the vpxd.cfg file is located in the C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter directory.
Use a text editor (such as WordPad) to open the vpxd.cfg file. The vpxd.cfg file is an XML-formatted text file that contains various optional settings for the VirtualCenter Server or vCenter Server system.
Add opening and closing tags to identify the hosts, versions, and other specifics to which the mask applies.
Copy and paste from the examples in Common Mask Patterns, or apply other masks as needed for your deployment.
When the appropriate changes are made, restart the VMware VirtualCenter Server Service to enable the new masking.

Common mask patterns

The following are example masks for several common CPU feature masks. Copy, paste, and modify them as required for the environment. VMware does not support or recommend using the masks shown below, because of the risk of application or guest operating system failure after migration.

SSE3

SSSE3

SSE3 and SSSE3 Combined

Reverting to the Default CPU Compatibility Bit Mask

Revert to the default vMotion compatibility constraints by reverting any changes made to the default mask. This can be accomplished by following the appropriate steps for the type of masking that was applied:

When using per-virtual-machine masks, reset any changed rows to the defaults by clicking Reset All to Default on the CPU Identification Mask dialog box.
When using global masks: Remove any added tags entered in the vpxd.cfg file and restart the VirtualCenter service.

Additional Information:

For translated versions of this article, see:

Spanish: Compatibilidad de la CPU para vMotion - Migraciones impedidas debido a una incompatibilidad de la CPU - Cómo invalidar máscaras (2016791)

日本語: vMotion の CPU 互換性 - CPU の不一致による移行の阻止 - マスクのオーバーライド方法 (2078408)
简体中文: vMotion CPU 兼容性 - 由于 CPU 不匹配，阻止迁移 - 如何覆盖掩码 (2040925)

Feedback

thumb_up Yes

thumb_down No

CPU vendor	<default-vendor>, <amd>, <intel>
CPU ID level	<level-0>, <level-1>, <level-80000000>, <level-80000001>
CPU register	<eax>, <ebx>, <ecx>, <edx>