Consolidating vSphere managed snapshots crashes the VM intermittently
Article ID: 379266
Updated On:
Products
Issue/Introduction
Occasionally when trying to consolidate the snapshots of a Virtual Machine, the VM crashes.
VMware.log shows that LWD Filter is active and is running out of memory:
2024-06-18T10:03:39.122Z In(05) vcpu-0 - LWD: LwdFilter_DiskRequirements: allocation limits:
2024-06-18T10:03:39.122Z In(05)+ vcpu-0 - static memory: 34551904
2024-06-18T10:03:39.122Z In(05)+ vcpu-0 - memory per disk MB: 81
2024-06-18T10:03:39.122Z In(05)+ vcpu-0 - memory per IO: 10996
2024-06-18T10:03:39.122Z In(05)+ vcpu-0 - global memory: 33555624
...
2024-06-18T10:03:39.135Z In(05) vcpu-0 - FiltLib: VMKPrivate_FiltModInitDiskInfo failed: "Admission check failed for memory resource" (195887233).
2024-06-18T10:03:39.135Z In(05) vcpu-0 - FiltLib: FiltLibAttachToFiltMod failed with error: "Operation failed" (1).
2024-06-18T10:03:39.135Z In(05) vcpu-0 - DISKLIB-LIB : DiskLibFiltLibInit: Failed to create filtLib context: Operation failed (334).
2024-06-18T10:03:39.135Z In(05) vcpu-0 - DISKLIB-LIB : DiskLibOpenInt: Failed to create filtLib context: Operation failed (334).
2024-06-18T10:03:39.136Z In(05) vcpu-0 - DISKLIB-VMFS : "/vmfs/volumes/<path>/<name>-flat.vmdk" : closed.
...
2024-06-18T10:11:32.167Z In(05) vcpu-0 - ConsolidateItemComplete: Online consolidation failed for disk node 'scsi0:0': Operation failed (5).
2024-06-18T10:11:32.167Z In(05) vcpu-0 - Checkpoint_Unstun: vm stopped for 48902 us
Environment
vSphere 7.x
Cause
The virtual machine is running out of overhead memory and crashing.
Resolution
Method 1 — Increasing the growth limit (in MB) of the cluster the VM resides in
vCenter Server can allocate some additional memory to a VM, for when that VM's memory is overutilized. This is done by specifying a configuration parameter for the cluster managed by vCenter that the VM is running in.
1. In the vpxd.cfg file in /etc/vmware-vpx on the vCenter Server appliance, add this configuration between the <vpxd> and the </vpxd> tags:
<cluster>
<VMOverheadGrowthLimit>6</VMOverheadGrowthLimit>
</cluster>2. Restart the vpxd service to apply the configuration change.
# service-control --restart vmware-vpxd3. The vmkernel.log (On ESXI /var/run/log/vmkernel.log) of each ESXI will show the change.
2024-09-24T14:25:00.578Z In(182) vmkernel: cpu7:2099403 opID=9c38d9b0)Config: 761: "VMOverheadGrowthLimit" = 6, Old Value: 0, (Status: 0x0)
Method 2 — Increasing the VM's memory allocation
NOTE: If the VM did not have Memory Hot Plug enabled, then the VM will have to be powered off before increasing the memory allocation.
1. Log in to vSphere Client (for vCenter Server) or the ESXi host web client of the host running the afflicted VM.
2. Verify that the account used to log in has the Virtual machine.Change Configuration.Change Memory privilege on the VM.
3. Right-click the desired VM and select "Edit Settings".
4. In the Virtual Hardware tab, expand the Memory field.
5. Increase the amount of memory assigned to the VM.
Additional Information
This configuration provides an increased growth margin in MB to virtual machine overhead memory. You can increase this amount to larger values if doing so further improves virtual machine performance.
Feedback
