After configuring memory tiering in a vSphere environment, powering on a VM results in the following error:
This type of VM is not supported when software memory tiering is enabled. Failed to start the virtual machine.
Module MemSchedEarly power on failed. An error occurred while parsing scheduler specific configuration parameters.
ESXi 9.0 hosts and ESXi 9.1 hosts using memory tiering.
In ESX 9.0 and earlier, there are several virtual machine parameters that are disallowed when running on a host with memory tiering enabled. This causes the VM to fail reporting the error, This type of vm is not supported when software memory tiering is enabled.
Refer to the table below for a list of unsupported settings
| VM Profile | VMX Options |
Low latency VM | sched.cpu.latencySensitivity = "high" |
Security VM | SEV VM: sev.enable = "True" SGX VM: sgxInfo.epcSize is non-zero TDX VM: tdx.enable = "True" |
Fault Tolerant VM | ftcpt.enabled = "True" |
Monster VM | VMs with more than 1 TB Memory and 128 vCPUs |
Nested Virtualization VM | vhv.enable = "True" |
This restriction is not limited to power-on — it also applies during migration. If a VM that is not supported with Memory Tiering is migrated to a host with Memory Tiering enabled, the migration will fail, because this VM type can only run on a host that does not have Memory Tiering enabled.
To work around the error in ESX 9.0 and earlier, change or disable the unsupported parameters on the virtual machine, (see the VM Profile table above). - To let these VM types coexist in the same environment as Memory Tiering, keep a subset of hosts in the cluster with Memory Tiering disabled, or move these VMs to a different cluster that does not have Memory Tiering enabled.
To resolve the error, upgrade to ESX 9.1, where these VM types are permitted to power on and migrate to a Memory Tiering-enabled host.
The following VM types are now supported to run on a physical 9.1 ESX host with Memory Tiering enabled but will still be disabled from participating in memory tiering.
Note: With Nested VMs, the only exception is VBS (Virtualization Based Security) VMs. VBS VMs can participate in memory tiering using a VMX parameter.
By default, Memory Tiering is disabled for a Nested Virtualization VM.
The vmx option sched.mem.enableNestedTiering can be set to TRUE to enable Memory Tiering for Nested VMs.
Before enabling Memory Tiering support for VBS VMs, review and test the nature of workloads that need to run in these VMs.
Customers are advised to follow the same guidelines as for a normal memory tiering deployment as listed in the section "Using and monitoring Memory Tiering" in the performance white paper, Memory Tiering Performance.
Broadcom does not support running nested hypervisors within a virtual machine on VMware vSphere ESX. See KB, Support for running ESXi as a nested virtualization solution.