ESXi host disconnected from the vCenter frequently due to the lack of buffer space for TCPIP

Products

• VMware vSphere ESXi 7.0

Issue/Introduction

• TCP IP heap memory exhaustion occurs due to heap fragmentation or low availability of memory
• There could be several symptoms due to lack of memory:
  • ESX host not accessible
  • Failure of some services
  • PSODs
  • Host disconnects
  • Etc.

Environment

VMware ESXi 7U3n or earlier

Cause

• Lack of heap memory due to memory fragmentation and/or configuration restricts memory expansion
• vmkernel.log will have the below logging::
  YYYY-MM-DDTHH:mm:ss.765Z cpu65:257021746)WARNING: Tcpip: 2420: socreate(type=2, proto=0) failed with error No buffer space available (55)
YYYY-MM-DDTHH:mm:ss.001Z cpu64:257027065)WARNING: Tcpip: 2420: socreate(type=1, proto=6) failed with error No buffer space available (55)
YYYY-MM-DDTHH:mm:ss.001Z cpu1:257026653)WARNING: Tcpip: 2420: socreate(type=1, proto=6) failed with error No buffer space available (55)
YYYY-MM-DDTHH:mm:ss.919Z cpu3:2098900)WARNING: Tcpip: 2420: socreate(type=1, proto=6) failed with error No buffer space available (55)

Resolution

• A fix is available in ESXi 7U3o.
• The issue is resolved by improved handling of memory to reduce fragmentation and also provide an option to allow increasing heap memory to a larger value. This will reduce the chances of memory failures.