SDK Applications intermittently disconnect from vCenter Server

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• SDK Applications intermittently disconnect from vCenter Server
• The vCenter Server logs contain the error:
  
  Connection to localhost:8085 failed with error class Vmacore::SystemException(Only one usage of each socket address (protocol/network address/port) is normally permitted. )
   
• TCP/IP Ports are left open in a LAST_ACK state
• VMware Consolidated Backup operations intermittently fail with the error:
  
  503 Server Unavailable
   
• vSphere Client connections to vCenter Server fail with the error:
  
  The server '123.123.123.123' could not interpret the client's request. (The remote server returned an error: (503) Server Unavailable.)"

Environment

VMware vCenter Server 4.0.x
VMware VirtualCenter 2.5.x
VMware vCenter Server 4.1.x
VMware vCenter Server 5.0.x

Resolution

The behavior occurs because ephemeral ports, temporary ports that are used for client server communications, are not closed upon use. Ephemeral ports are limited to a range of ports and are only valid for the duration of the connection.

 

In this case, on certain Microsoft Windows operating systems, SDK connections leave the ports in a LAST_ACK state on the vCenter Server and therefore the range of ports is exhausted after a period of time. When this happens, SDK connectivity fails until a port is freed.

 

This is not a VMware issue. This issue is caused by a race condition with the Microsoft TCP/IP driver. 

 

Note: The preceding link was correct as of March 25, 2010. If you find the link is broken, provide feedback and a VMware employee will update the link.