running out of SSH connection

Products

• CA API Gateway

Issue/Introduction

Clients connecting to gateway via SFTP failing when pool size is reached.  Looking to increase the pool size

Error ssg log:

2021-05-15T02:15:28.023-0400 SEVERE  1667 com.l7tech.external.assertions.ssh.server.ae: Exception caught in SFTP subsystem
java.util.concurrent.RejectedExecutionException: Task [email protected] rejected from [email protected][Running, pool size = 20, active threads = 20, queued tasks = 25, completed tasks = 118]

I did a total count of SSH connection, and each gateway shows 124 connections. Here is our setting in cluster property

Below are parameter settings we implemented but none of them change the pool size 

ssh.responseDownloadThreadLimit = 100

ssh.session.pool.maxActive = 500

ssh.routingInactiveTinmeout = 10s

 

Environment

Release : 10

Component : API GATEWAY

Resolution

You can add then adjust hidden CWP sftp.messageProcessingThreadLimit (pool size) - global limit of processing threads for SFTP requests (default 20)

 

sftp.messageProcessingThreadLimit 100 

 

I performed a test that over loaded SSH2 assertions over 100 concurrent requests 

 

(NOTE: the pool size now reflects the large amount of threads) 

2021-06-18T14:30:25.670-0400 WARNING 1136 com.l7tech.external.assertions.ssh.server.ServerSshRouteAssertion: 9434: SSH routing error: Task [email protected] rejected from [email protected][Running, pool size = 100, active threads = 3, queued tasks = 25, completed tasks = 14892]