Error: No connections available for Found In-Use = true in SPS
Article ID: 240160
Updated On:
Products
SITEMINDER
CA Single Sign On Secure Proxy Server (SiteMinder)
Issue/Introduction
When running CA Access Gateway (SPS), this one reports an error:
No connections available for [{s}->https://mysps.mydomain.com:443] Found In-Use = true]
This error occurs with the integration of this specific application.
Environment
CA Access Gateway (SPS) 12.8SP4 on RedHat 7
Cause
The SPS get started at 04:51:54:
sps-webagent.log:
[1860/4304][Sun Apr 03 2022 04:51:54] CA Access Gateway Agent, Version 12.8 QMR04, Update None, Label 2278
sps-webagent-trace.log:
[04/03/2022][04:51:54][1860][4304][][CSmLowLevelAgent.cpp:6279][InitializeTracingMT]
[][][][][][][][][][][][][][][][Tracing initialized.]
and the first error occurrence appears at 05:12:32 :
[04/03/2022][05:12:32][1860][8812][16f9d0a3-ede8aa60-efdf14f1-d03e9208-fc993223-77d7]
[com.ca.proxy.connectionpool.impl.AuthNConnectionPoolImpl][getConnection][][][][][]
[][][][][][][][][][]
[No connections available for [{s}->https://mysps.mydomain.com:443] Found In-Use = true]
Before that first occurrence, the SPS already has intermittent problem
to get connection with the backend server, as with
myothersps.mydomain.com for instance, which failed after 3 tentatives:
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[ProxyValve.java][ProxyValve::invoke][][][][][][][][][][][][][][][][Entering the agent.]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[CSmHttpPlugin.cpp:869][CSmHttpPlugin::ProcessResource][][][][agent][][][][][][][][]
[][][][Resolved URL: '/myapp/myapp.html'.]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[CSmHttpPlugin.cpp:1001][CSmHttpPlugin::ProcessResource][][[email protected] ][][agent]
[/myapp/myapp.html][][][][][][][][][][][Resolved METHOD: 'GET'.]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[SmAgentAPI.cpp:1761][Sm_AgentApi_IsProtected][][https://myothersps.mydomain.com][][agent]
[/myapp/myapp.html][][][][][][][][][][][]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[CSmLowLevelAgent.cpp:577][IsResourceProtected][][*][][agent]
[/myapp/myapp.html][][][][][][][][][][][Resource is not protected from Policy Server.]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[SmProxyRules.java][SmProxyRules.processRules][][][][][][][][][][][][][][][]
[Dispatching to service FORWARD with url https://10.0.0.1/myapp/myapp.html]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Sending request to backend = 10.0.0.1 url = https://10.0.0.1/myapp/myapp.html]
[04/03/2022][04:58:37][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.conn.factory.SPSConnectionFactory][openConnection()]
[][][][][][][][][][][][][][][][Connecting to /10.0.0.1:443]
[04/03/2022][04:58:47][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Retrying to send the request to backend web server.Retry count: 1]
[04/03/2022][04:58:47][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Sending request to backend = 10.0.0.1 url = https://10.0.0.1/myapp/myapp.html]
[04/03/2022][04:58:47][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.conn.SPSConnectionManager][requestConnection(): ][]
[][][][][][][][][][][][][][][Get connection: {s}->https://10.0.0.1:443, timeout = 180000]
[04/03/2022][04:58:57][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Retrying to send the request to backend web server.Retry count: 2]
[04/03/2022][04:58:57][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Sending request to backend = 10.0.0.1 url = https://10.0.0.1/myapp/myapp.html]
[04/03/2022][04:58:57][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.conn.SPSConnectionManager][requestConnection(): ][]
[][][][][][][][][][][][][][][Get connection: {s}->https://10.0.0.1:443, timeout = 180000]
[04/03/2022][04:58:57][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.conn.factory.SPSConnectionFactory][openConnection()]
[][][][][][][][][][][][][][][][Connecting to /10.0.0.1:443]
[04/03/2022][04:59:07][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Retrying to send the request to backend web server.Retry count: 3]
[04/03/2022][04:59:07][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Tried to send the request to backend web server three times.Throwing the exception to client. ]
[04/03/2022][04:59:07][1860][3004][78cf9c61-d361cfae-9a61b3c9-8c0c7484-25852099-eff]
[Noodle.java][Noodle::doGet][][][][][][][][][][][][][][][]
[org.apache.http.client.ClientProtocolException
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:839)]
[04/03/2022][04:59:08][1860][3004][18155d83-b7d04b90-9c585ec4-17959c27-f35ea6e7-ad40]
[ProxyValve.java][ProxyValve::invoke][][][][][][][][][][][][][][][][Entering the agent.]
So going on that way, SPS needs to consistently allocate new
connections until it reaches the limit and we see the error:
No connections available for
This backend server doesn't have always connectivity problems. At
04:58:37, the myothersps.mydomain.com / 10.0.0.1 replies within the
same second.
[04/03/2022][04:58:37][1860][8952][1a03da5d-3a3761c5-799d6ba0-7bf317fd-fa6ca99d-8dd]
[SmProxyRules.java][SmProxyRules.processRules][][][][][][][][][][][][][][][]
[Proxy Rules Debug: Evaluating Condition: Type: null, Case: 'myothersps.mydomain.com:443'
equals 'myothersps.mydomain.com:443']
[04/03/2022][04:58:37][1860][8952][1a03da5d-3a3761c5-799d6ba0-7bf317fd-fa6ca99d-8dd]
[SmProxyRules.java][SmProxyRules.processRules][][][][][][][][][][][][][][][]
[Dispatching to service FORWARD with url https://10.0.0.1/myotherapp/myotherapp.html]
[04/03/2022][04:58:37][1860][8952][1a03da5d-3a3761c5-799d6ba0-7bf317fd-fa6ca99d-8dd]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Sending request to backend = 10.0.0.1 url = https://10.0.0.1/myotherapp/myotherapp.html]
[04/03/2022][04:58:37][1860][8952][1a03da5d-3a3761c5-799d6ba0-7bf317fd-fa6ca99d-8dd]
[com.ca.proxy.apache.httpclient.conn.SPSConnectionManager][requestConnection(): ][]
[][][][][][][][][][][][][][][Get connection: {s}->https://10.0.0.1:443, timeout = 180000]
[04/03/2022][04:58:37][1860][8952][1a03da5d-3a3761c5-799d6ba0-7bf317fd-fa6ca99d-8dd]
[com.ca.proxy.apache.httpclient.conn.factory.SPSConnectionFactory][openConnection()]
[][][][][][][][][][][][][][][][Connecting to /10.0.0.1:443]
[04/03/2022][04:58:37][1860][8952][1a03da5d-3a3761c5-799d6ba0-7bf317fd-fa6ca99d-8dd]
[com.ca.proxy.apache.httpclient.SPSClient][execute][][][][][][][][][][][][][][][]
[Response status code from backend webserver is 200]
Resolution
The connectivity problem is intermittent, and not only limited to
mysps.mydomain.com.
Even before the first error occurrence, a bunch of connections
tentative fail, as the SPS tries many times to reach the backend
server without success. But not always! Some connections are done
quickly, but not all!
When restarting SPS, the entire connection pool is available, but it
becomes busy within 20 minutes because it accumulates the calls which
take a long time to get a connection with the backend servers.
To fix this issue, you have to investigate the network and the backend
servers' responsiveness and connectivity.
Feedback
Yes
No
Powered by
