vCenter Server Appliance Management Interface or VAMI login fails with error "[Errno 24] Too many open files" and SSH is unresponsive

Products

• VMware vCenter Server

Issue/Introduction

The vCenter Server Appliance Management Interface (VAMI) on port 5480 becomes unavailable.
Users are unable to log in via the UI, receiving an explicit error:
Exception in invoking authentication handler [Errno 24] Too many open files.

As a side effect of this resource exhaustion, SSH sessions to the vCenter Server become unresponsive or hang during login.

Symptoms:
- VAMI (https://:5480) login fails with the "Too many open files" exception.
- SSH to the vCenter Server is unresponsive.
- The applmgmt service appears hung or overloaded.
- applmgmt.log contains repeated entries:

/var/log/vmware/applmgmt/applmgmt.log

<DATE && TIME> [9933]ERROR:vmware.appliance.syslog.syslog_lib:Got exception in _getIPEb, failure = [Failure instance: Traceback: <class 'OSError'>: [Errno 24] Too many open files
/usr/lib/applmgmt/syslog/py/vmware/appliance/syslog/syslog_lib.py:209:_checkConnection
/usr/lib/applmgmt/syslog/py/vmware/appliance/syslog/syslog_lib.py:173:_check
/usr/lib/python3.11/site-packages/twisted/internet/defer.py:532:addCallback
/usr/lib/python3.11/site-packages/twisted/internet/defer.py:512:addCallbacks
--- <exception caught here> ---

Environment

vSphere 9.0.X

 

Cause

The issue is caused by a socket connection leak within the applmgmt process.
Continuous syslog API messages sent to applmgmt result in accumulated socket connections (specifically targeting port 514) that are not properly cleared.
When the process reaches the Linux default limit of 1024 open files, it can no longer open new files or sockets.
This breaks VAMI authentication, and causes SSH to become unresponsive due to process overload.

This can be confirmed via, teh steps from below 

1. GET applmgmt process ID
# pgrep -alf applmgmt | grep serve
<PID> /usr/bin/python /usr/lib/applmgmt/base/bin/vherdrunner /usr/lib/applmgmt/transport/bin/serve

2. use Socket count utility and grep for port 514 connections used by syslog 
# ss -pant | grep <PID> | grep 514
See https://explainshell.com/explain?cmd=ss+-pant+%7C+grep+1234+%7C+grep+514 

Resolution

VMware Engineering is aware about thsi, and will address the issue in a future release. 
For the workaround, please reach out to Broadcom support and reference this KB - 430003