vMotion fail with "Module 'VMMon' power on failed. "

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• vMotion fail and "Cannot migrate VM Name from source Host, xxxx in xxx to target Host, xxxx in xxxx" on Event
• In /var/log/vmware/vpxd/vpxd.log, you see entries similar to:
  
  YYYY-MM-DDTHH:MM:SS.MSZ error vpxd[05602] [Originator@6876 sub=VmProv opID=CdrsLoadBalancer-2d3678fd-6d9c3c12-01-01] Get exception while executing action vpx.vmprov.InitiateDestination:
  --> (vmodl.fault.SystemError) {
  -->    faultCause = (vim.fault.GenericVmConfigFault) {
  -->       faultMessage = (vmodl.LocalizableMessage) [
  -->          (vmodl.LocalizableMessage) {
  -->             key = "msg.moduletable.powerOnFailed",
  -->             arg = (vmodl.KeyAnyValue) [
  -->                (vmodl.KeyAnyValue) {
  -->                   key = "1",
  -->                   value = "VMMon"
  -->                }
  -->             ],
  -->             message = "Module 'VMMon' power on failed. "
  -->          },
  -->          (vmodl.LocalizableMessage) {
  -->             key = "msg.migrate.resume.fail",
  -->             message = "The VM failed to resume on the destination during early power on. ",
  -->          },
  -->          (vmodl.LocalizableMessage) {
  -->          (vmodl.LocalizableMessage) {
  
  
• In /var/log/hostd.log on Target ESXi host, you see entries similar to:
  

  
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102429]: [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/datastore/xxxxxxxx/vmname.vmx] VMotion destination started; powering on
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102415]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore/xxxxxxxx/vmname.vmx] Handling vmx message 

  : Group vm.88793567: Cannot admit virtual machine: CPU admission check failed. Invalid reservation 1000 mhz.
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102393]: --> Group host/user: Invalid CPU allocation requested for virtual machine vmx. (min: 1000 mhz, max: 4294967295, minLimit: 4294967295, shares: 4294967293)
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102393]: --> Could not power on virtual machine: Admission check failed for cpu resource.
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102393]: --> See the VMware ESX Resource Management Guide for information on resource management settings.
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102393]: --> Failed to power on VM.
  YYYY-MM-DDTHH:MM:SS.MSZ Db(167) Hostd[2102393]: -->
  YYYY-MM-DDTHH:MM:SS.MSZ Wa(164) Hostd[2102415]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore/xxxxxxxx/vmname.vmx] Failed to find activation record, event user unknown.
  YYYY-MM-DDTHH:MM:SS.MSZ In(166) Hostd[2102415]: [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 1187224 : Error message on vmname on esxi hostname in ha-datacenter: Failed to power on VM.

• In vmware.log, you see entries similar to:

         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - Msg_Post: Error
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - [vob.sched.group.cpu.admitfailed] Group vm.123823645: Cannot admit virtual machine: CPU admission check failed. Invalid reservation 6000 mhz.
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - [vob.sched.group.cpu.invalid] Group host/user: Invalid CPU allocation requested for virtual machine vmx. (min: 6000 mhz, max: 4294967295, minLimit: 4294967295, shares: 4294967293)
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - [msg.vmmonVMK.admitFailed] Could not power on virtual machine: Admission check failed for cpu resource.
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - See the VMware ESX Resource Management Guide for information on resource management settings.
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - [msg.monitorLoop.createVMFailed.vmk] Failed to power on VM.
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - ----------------------------------------
         YYYY-MM-DDTHH:MM:SS.MSZ Wa(03) vmx - VMMon_PowerOn: Failed to create VM instance: Admission check failed for cpu resource
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - Module 'VMMon' power on failed.
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - VMX_PowerOn: ModuleTable_PowerOn = 0
         YYYY-MM-DDTHH:MM:SS.MSZ In(05) vmx - MigrateSetStateFinished: type=2 new state=MIGRATE_FROM_VMX_FINISHED

Environment

• ESXi 7.x
• ESXi 8.x
• ESX 9.x
• vCenter 7.x
• vCenter 8.x
• vCenter 9.x

Cause

This issue happen when the virtual machine's CPU or memory reservation is set, but the target Esxi host does not have sufficient resources to allocate the reserved resource.

Resolution

To resolve this issue, either reduce the CPU/memory reservation or remove the reservation