UIM Linux Hub probe restarts by itself; SSL alert (write): fatal: decode error

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The hub probe intermittently restarts on a secondary hub (Proxy hub) with 30+ hubs connecting in as tunnel clients. This secondary also connects to the Primary hub as tunnel client.

To confirm if the hub probe is restarting,  open the Hub probe config in IM and check the Uptime in the General tab.

The hub log on the Primary hub with loglevel 4 captures the following before the secondary hub restarts

hub: SSL alert (write): fatal: decode error
[Proxy1A] - SSL layer error (1) encountered, ending session
tsess_read_message: ts is done; state=[0x2c]. Returning SSL_CLOSE
hub: TSESS-A-4177 marked for removal (reason: TsessSslClose)

 

The controller log on the secondary hub with loglevel 3 captures the entries below when the hub restarts

Controller: Process stopped (probe=hub, pid=1579065) SIG = 139
Controller: _ProcStart - Probe 'hub' - starting

 

Note: The hub log on the secondary hub with loglevel 4 does not capture any errors when it restarts. A new hub.log is created on the restart.

Environment

UIM 23.4.0

Hub probe 23.4.0

Resolution

This issue is seen when the Hub goes beyond the default file descriptor limit and is unresponsive to controller.  The controller stops the hub and restarts again which causes queue disconnection/connection again and again.

To set the file descriptor with unlimited sessions, add the 'tunnel_max_fd = 0' key in the secondary hub hub.cfg under hub section

Additional Information

UIM Linux Hub restarts by itself: caused hub to reach max file descriptors

hub: nimSessionWait: epoll_handle: -1 failed EPOLL_CTL_ADD fd:1004 error: 9 error Str: Bad file descriptor