Copy on Write (COW) Heap limitations for ESXi 4.1, 5.x and 6.0
search cancel

Copy on Write (COW) Heap limitations for ESXi 4.1, 5.x and 6.0

book

Article ID: 343426

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information on the limitations of Copy on Write (COW) Heap.



Symptoms:

  • Copy on Write (COW) out of memory.
  • You see these errors within the vmkernel log files:

    WARNING: Cow: 549: No memory available

    Mar 14 13:15:29 vmkernel: 7:01:14:51.338 cpu16:2366427)World: vm 2374623: 1540: Starting world vmm0:VMNAME with flags 4008
    Mar 14 13:15:29 vmkernel: 7:01:14:51.338 cpu16:2366427)Sched: vm 2374623: 6054: Adding world 'vmm0:VMNAME', group 'host/user', cpu: shares=-3 min=0 minLimit=-1 max=-1, mem: shares=-3 min=0 minLimit=-1 max=-1
    Mar 14 13:15:29 vmkernel: 7:01:14:51.338 cpu16:2366427)Sched: vm 2374623: 6073: renamed group 379434 to vm.2366427
    Mar 14 13:15:29 vmkernel: 7:01:14:51.338 cpu16:2366427)Sched: vm 2374623: 6090: group 379434 is located under group 4
    Mar 14 13:15:29 vmkernel: 7:01:14:51.350 cpu16:2366427)Swap: vm 2374623: 2331: extending swap to 4194304 KB
    Mar 14 13:15:29 vmkernel: 7:01:14:51.399 cpu23:2366427)World: vm 2366433: 1540: Starting world vmm1:VMNAME with flags 4008
    Mar 14 13:15:29 vmkernel: 7:01:14:51.764 cpu17:2366427)WARNING: Cow: 549: No memory available! Called from 0x41800dc39a69
    Mar 14 13:15:29 vmkernel: 7:01:14:51.764 cpu17:2366427)Cow: 1759: Failed on handle 3 (7344784) of 4 with Out of memory
    Mar 14 13:15:29 vmkernel: 7:01:14:51.764 cpu17:2366427)WARNING: Cow: 1329: Opening the cow hierarchy for 46990d32-VMNAME_5-delta.vmdk failed with Out of memory
    Mar 14 13:15:29 vmkernel: 7:01:14:51.955 cpu17:2366427)Sched: vm 2374623: 1487: name='vmm0:VMNAME
    Mar 14 13:15:29 vmkernel: 7:01:14:51.955 cpu17:2366427)CpuSched: vm 2374623: 9990: zombified unscheduled world: runState=NEW
    Mar 14 13:15:29 vmkernel: 7:01:14:51.955 cpu17:2366427)World: vm 2374623: 4024: deathPending set; world not running, scheduling reap
    Mar 14 13:15:29 vmkernel: 7:01:14:51.955 cpu17:2366427)Sched: vm 2366433: 1487: name='vmm1:VMNAME'
    Mar 14 13:15:29 vmkernel: 7:01:14:51.955 cpu17:2366427)CpuSched: vm 2366433: 9990: zombified unscheduled world: runState=NEW
    Mar 14 13:15:29 vmkernel: 7:01:14:51.955 cpu17:2366427)World: vm 2366433: 4024: deathPending set; world not running, scheduling reap



Environment

VMware vSphere ESXi 5.0
VMware vSphere ESXi 5.1
VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.5
VMware vSphere ESXi 6.0
VMware ESXi 4.1.x Installable

Resolution

On average, 20% of the COW Heap Memory is used for internal data structure. This means that 80% of the memory can be used for virtual machines. Since root entries mainly occupy the memory, 75% of the memory is used for root entries.

The maximum configurable size for COW Heap Memory is 256 MB. The default configurable size is 192 MB.
 
For example:
Z = Number of Virtual Machine's
Y = Number of disks
X = Number of snapshotsthat each disk has
W = Size (in bytes)

Z = (75 / 100 * COW_HEAP_SIZE) / ((W / (2 * 1048576) * 4 * X) * Y), where 2 * 1048576 is GDE Coverage and constant 4 is bytes per Root Entry.
 
If a Virtual Machine has 5 disks, each disk is 80 GB (85899345920 bytes) with 6 levels of snapshots for each disk. The number of Virtual Machines that can be powered-on the ESX host on which the COW Heap size is configured as 256 MB is

= (75 / 100 * 268435456) / ((85899345920 / (2 * 1048576) * 4 * 6) * 5)
Approximately 40 of such Virtual Machines can be powered on.
Note: Use the equation above to calculate your COW requirements and determine your limitations.
 



Additional Information