Virtual machines lose network connectivity after a vMotion or crash with "reservation failed" errors

Products

• VMware vSphere ESXi

Issue/Introduction

• A virtual machine might fail with the following error message in vmware.log:
  
  YYYY-MM-DDTHH:MM:SS.sssZ| vcpu-1| W115: MONITOR PANIC: vmk: vcpu-0:P2M reservation failed after max retries
  
  
• A virtual machine might fail to enable its network adapter after a vMotion with the following error messages in vmkernel.log
  
  YYYY-MM-DDTHH:MM:SS.sssZ cpu174:4463989)VmMemCow: 1706: p2m update: cannot reserve - cur 0 0 rsvd 1028 req 257 avail 1279
YYYY-MM-DDTHH:MM:SS.sssZ cpu174:4463989)Vmxnet3: 16524: Failed to map the rx data ring for rq 0

Note: NSX Edge VMs can be affected by this issue

Environment

VMware vSphere ESXi 6.x, 7.x, 8.x, and 9.x
VMware NSX version 4.0.x or Above (if applicable)

Cause

A virtual machine has requested a larger memory size for a VMX device (ex: SVGA, 3D, etc.), this may require breaking up a large number of shared memory pages causing the p2m (Physical Page Number to Machine Page Number) buffer to overflow.  The virtual machine may power off with the error message "reservation failed after max retries", or become disconnected from the network.

Resolution

To workaround this issue, you need to increase the P2M buffer slots on the ESXi host.

• The host does not require a reboot after making this change.
• The new setting will only take effect on virtual machines when they are powered on or migrated onto the host after the setting has been applied.

Implementation Steps:

1. Before changing the setting, you must either:
   • Power off the affected virtual machine(s) on the ESXi host.
   • Migrate any powered-on virtual machines to another host in the cluster.
2. In the vSphere Client, navigate to Host > Configure > System > Advanced System Settings.
   
3. Choose Edit...
4. Search for Mem.ShareCOSBufSize.
   
5. Change the value to 32 (this is the maximum supported size).
6. Click OK to apply the new value.
7. After the setting is updated, you can:
   • Power on the previously powered-off virtual machine(s).
   • Migrate the virtual machine(s) back to this ESXi host (if you moved them).

Note:  If the VM logging the messages is an NSX Edge VM, consider upgrading the setting on the following combination of products. Change the value on hosts which already have a set ShareCOSBufSize to 32 :

• NSX - 4.1.0 or above
• ESX - 8.0 or above