ESXi 5.1/5.5 host is disconnected in vCenter Server and reports the error: No buffer space available

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• ESXi 5.1 or ESXi 5.5 host is disconnected in vCenter Server.
• Local datastore on the host is marked as inaccessible.
• Unable to connect directly to the host using the vSphere Client.
• The host's management network IP responds to ping.
• The hostd management agent is running.
• Restarting the management agents fails with the error:
  
  Connect to localhost failed: Connection failure
   
• In the /var/log/vmkernel.log file, you see an entry similar to:
  
  WARNING: Tcpip: 1304: socreate(type=1, proto=6) failed with error No buffer space available (55)
   
• In the /var/log/vpxa.log file, you see entries similar to:
  
  [73C59B90 verbose 'hostdcnx'] [VpxaHalCnxHostagent] Creating temporary connect spec: localhost:443
  [73C17B90 error 'HttpConnectionPool-000000'] [ConnectComplete] Connect failed to <cs p:0db52800, TCP:localhost:443>; cnx: (null), error: N7Vmacore15SystemExceptionE(Connection reset by peer)
  [73C59B90 error 'httphttpUtil'] [HttpUtil::ExecuteRequest] Error in sending request - Connection reset by peer
  [73C59B90 error 'hostdcnx'] [VpxaHalCnxHostagent] Failed to discover version: vim.fault.HttpFault
  [73C59B90 warning 'hostdcnx'] [VpxaHalCnxHostagent] Could not resolve version for authenticating to host agent
  [73C38B90 verbose 'hostdcnx'] [VpxaHalCnxHostagent] Creating temporary connect spec: localhost:443
  [FFB99B90 error 'HttpConnectionPool-000000'] [ConnectComplete] Connect failed to <cs p:0db60350, TCP:localhost:443>; cnx: (null), error: N7Vmacore15SystemExceptionE(Connection reset by peer)
  [73C38B90 error 'httphttpUtil'] [HttpUtil::ExecuteRequest] Error in sending request - Connection reset by peer

Cause

This issue occurs if applications do not close their TCP socket connections properly.

Resolution

This issue is resolved in ESXi 5.5 Update 1.

This is a known issue in ESXi 5.1. Currently, there is no resolution or workaround.

Additional Information

If a host becomes inaccessible, its virtual machines continue to run and have network connectivity. You may have to remotely connect (through RDP or VNC) to each virtual machine and power them off.