ESXi host disconnects from vCenter with error: No buffer space available (55)
search cancel

ESXi host disconnects from vCenter with error: No buffer space available (55)

book

Article ID: 440355

calendar_today

Updated On:

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

Symptoms:

  • One or more ESXi hosts intermittently disconnect from vCenter Server.
  • The host becomes unreachable via management (SSH, UI) and management agents may fail to restart.
  • Virtual machines on the affected host may lose network connectivity or manageability.
  • The issue is temporarily resolved only by a physical reboot of the ESXi host.
  • The vmkernel.log located at /var/run/log/ will have entries similar to the following:
    WARNING: Tcpip: 2440: socreate(type=1, proto=6) failed with error No buffer space available (55)
    WARNING: Hbr: Failed to create socket to 127.0.0.1: No buffer space available
    WARNING: vdl2: Failed to refresh gateway MAC, vmknic [vmk10]/[vmk11]: Status pending


Environment

VMware SDDC Manager 5.2.x

VMware ESXi Host 8.x

Cause

This issue occurs due to TCP/IP heap memory exhaustion or fragmentation in the ESXi vmkernel. This is most common in environments running VMware NSX or vSphere Replication (HBR), where multiple network stacks and frequent socket creation/deletion can exceed the default heap allocation.

In some cases, this is exacerbated if IPv6 is disabled while IPFIX is enabled, as the system continuously attempts and fails to create IPv6-based sockets, rapidly consuming available buffer space.

Resolution

A permanent fix for this memory allocation and fragmentation issue is included in ESXi 8.0 Update 3i (Build 24585300) and later. For more information, see the ESXi 8.0 Update 3i Release Notes.

Workaround: If you cannot immediately upgrade to ESXi 8.0 Update 3i, perform the following steps to mitigate the issue:

  1. Increase TCP/IP Heap Size:
    Increasing the heap size allows the vmkernel to manage more simultaneous socket connections.
    esxcfg-advcfg -s 128 /Net/TcpipHeapSize

    Note: A host reboot is required for this change to take effect.


  2. Verify IPv6 and IPFIX Configuration:
    If IPv6 is disabled on ESXi hosts, ensure that IPFIX (NetFlow) is not enabled in the NSX environment, or enable IPv6 on the ESXi hosts. Using this command below to check:
    esxcli network ip set --ipv6-enabled=true

Additional Information