Shared-nothing vMotion of a VM to another cluster fails with error 'The available Memory resources in the parent resource pool are insufficient for the operation'
search cancel

Shared-nothing vMotion of a VM to another cluster fails with error 'The available Memory resources in the parent resource pool are insufficient for the operation'

book

Article ID: 315464

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

To be able to live migrate a VM from one cluster to another.


Symptoms:

Shared-nothing(both compute and storage vMotion) vMotion of a VM to another cluster fails with error 'The available Memory resources in the parent resource pool are insufficient for the operation'

 

vpxd.log

2023-12-11T06:08:23.178Z warning vpxd[10143] [Originator@1234 sub=obn opID=<opID>] post migrate callback is skipped - VM migration failed

 

2023-12-11T06:08:23.178Z error vpxd[10143] [Originator@1234 sub=vmpro999 opID=<opID>] Get exception while executing action vpx.vmprov.InvokeCallbacks: N3Vim5Fault32InsufficientMemoryResourcesFault9ExceptionE(Fault cause: vim.fault.InsufficientMemoryResourcesFault

--> )

 

2023-12-11T06:08:23.184Z info vpxd[10143] [Originator@1234 sub=vmpro999 opID=<opID>] Workflow context:

 

2023-12-11T06:08:23.230Z info vpxd[10143] [Originator@1234 sub=Default opID=<opID>] [VpxLRO] -- ERROR task-2069600 -- vm-12345 -- vim.VirtualMachine.relocate: vim.fault.InsufficientMemoryResourcesFault:

--> Result:

--> (vim.fault.InsufficientMemoryResourcesFault) {

--> faultCause = (vmodl.MethodFault) null,

--> faultMessage = <unset>,

--> unreserved = 1393831182336,

--> requested = 1125899906318336

--> msg = ""

--> }

--> Args:

-->

--> Arg spec:

--> (vim.vm.RelocateSpec) {

--> service = (vim.ServiceLocator) null,

--> folder = <unset>,

--> datastore = 'vim.Datastore:777776f8-b99c-4230-a820-7f0ea1311111:datastore-22222',

--> diskMoveType = <unset>,

--> pool = 'vim.ResourcePool:777776f8-b99c-4230-a820-7f0ea1311111:resgroup-11111',

--> host = 'vim.HostSystem:777776f8-b99c-4230-a820-7f0ea1311111:host-33333',

--> disk = (vim.vm.RelocateSpec.DiskLocator) [

--> (vim.vm.RelocateSpec.DiskLocator) {

 

Environment

VMware vCenter Server 7.0.x

Cause

The large required value is OMM_MAX_OVERHEAD_BYTES, which is set when we do not have a valid memory overhead limit for a vm/host pair.

 

Based on the vpxd logs, it appears OMM could not parse the vmxConfig for vm-12345 successfully due to spaces in some custom keys in the vmdk descriptor.

 

2023-12-11T05:45:28.035Z warning vpxd[07699] [Originator@1234 sub=OMM opID=<opID>] ConfigParser: parse error near key #disk0.ddb.customVolumeMetadataName\\Hard

2023-12-11T05:45:28.035Z warning vpxd[07699] [Originator@1234 sub=OMMVmInfo opID=<opID>] Failed to parse vmxConfig for VM [vim.VirtualMachine:vm-12345

 

Looking at the vmdk files for this VM in esx log bundle, we see these key/value pairs with spaces in the key in all 3 vmdks.

 

disk4000.vmdk

 

ddb.customVolumeMetadataName\\Hard disk 1\Volume 1 System Reserved = "77777##########12345:6705"

ddb.customVolumeMetadataName\\Hard disk 1\Volume 2 = "88888#########54321:6705"

 

disk4001.vmdk

 

ddb.customVolumeMetadataName\\Hard disk 2\Volume 1 Pagefiles = 999999#########11111:6705"

 

disk4002.vmdk

 

ddb.customVolumeMetadataName\\Hard disk 3\Volume 1 New Volume = "666666########9456:6705"

Resolution

1. Take a backup of the VM/take a copy of the files to be modified.

2. Shut down the VM.

3. Remove the below lines from the respective files.

> disk4000.vmdk

 

ddb.customVolumeMetadataName\\Hard disk 1\Volume 1 System Reserved = "77777##########12345:6705"

ddb.customVolumeMetadataName\\Hard disk 1\Volume 2 = "88888#########54321:6705"

 

> disk4001.vmdk

 

ddb.customVolumeMetadataName\\Hard disk 2\Volume 1 Pagefiles = 999999#########11111:6705"

 

> disk4002.vmdk

 

ddb.customVolumeMetadataName\\Hard disk 3\Volume 1 New Volume = "666666########9456:6705"

 

4. Power on the VM.

5. Perform vMotion.


Workaround:

Not applicable

 

Additional Information

To view the vmdk file, use below command.

cat disk4000.vmdk

 

Find the lines similar to below:

ddb.customVolumeMetadataName\\Hard disk 1\Volume 1 System Reserved = "55555###########1654:6705"

 

Remove the line using vi editor.

vi disk4000.vmdk


Impact/Risks:

Unable to live migrate VM to another cluster