Agent Api function failed when load balancer is introduced between agent and policy server
Article ID: 38141
Updated On:
Products
Issue/Introduction
Introducing a load balancer between the agent and the Policy server can cause Commutation failure errors that can be identified in the Agent Log starting with -2 followe by -1 Error as seen below
[14259/1151969248][Sun Feb 07 2016 12:58:21][CSmLowLevelAgent.cpp:546][ERROR][sm-AgentFramework-00520] LLA:SiteMinder Agent Api function failed - 'Sm_AgentApi_IsProtectedEx' returned '-2'.
[14257/1151969248][Sun Feb 07 2016 13:06:58][CSmProtectionManager.cpp:192][ERROR][sm-AgentFramework-00420]HLA: Component reported fatal error: 'Low Level Agent'.
[14257/1151969248][Sun Feb 07 2016 13:06:58][CSmHighLevelAgent.cpp:413][ERROR][sm-AgentFramework-00420] HLA:Component reported fatal error: 'Protection Manager'.
[14257/1151969248][Sun Feb 07 2016 13:07:55][CSmLowLevelAgent.cpp:1378][ERROR][sm-AgentFramework-00520] LLA:SiteMinder Agent Api function failed - 'Sm_AgentApi_LoginEx' returned'-1'.
[14257/1151969248][Sun Feb 07 2016 13:17:10][CSmLowLevelAgent.cpp:1378][ERROR][sm-AgentFramework-00520] LLA:SiteMinder Agent Api function failed - 'Sm_AgentApi_LoginEx' returned'-1'.
[14257/1151969248][Sun Feb 07 2016 13:17:10][CSmAuthenticationManager.cpp:194][ERROR][sm-AgentFramework-00420] HLA: Component reported fatal error: 'Low Level Agent'.
[14257/1151969248][Sun Feb 07 2016 13:17:10][CSmHighLevelAgent.cpp:1244][ERROR][sm-AgentFramework-00420] HLA: Component reported fatal error: 'Authentication Manager'
Environment
Component:
Resolution
As you have introduced a load balancer between the Agent and the Policy server ,here is what it can happen
- the Policy server has a TCP Idle Timeout value set to 10 min (default) which is configurable from the smconsole . What that means is that the Policy server will timeout an active connection from agent if no request received on it within 10 min .
- When a load balancer is introduced in the middle ,most of these load balancers has the session timeout set to 5 min hence the problem .
The steps below explains how the problem occurs
1) Agent open a connection to the policy server where Normal priority requests are served
2) If the agent does not send any request on this connection within 5 min ,the load balancer will timeout the session and close the connection
3) The problem arises as the load balancer does not notify the Agent nor the policy server of this closed connection
4) For the policy server ,it will terminate the connection from its end after another 5 min (total 10 min) as no connection was received from Agent within the last 10 min. The policy server will notify the agent that connection was closed .
5) The problem is that the load balancer receives the close connection from Policy server ,however ,as the load balancer already dropped the connection as explained in Step 3 ,the close request never reaches the agent
6) now the agent receives requests that he needs to process ,he looks for the available connections in the pool and as the connection in question is still available ,the agent will process the call and by default will wait for 60 seconds to hear back from the policy server .
7) The load balancer receives the request from agent and as it dropped already the connection ,it will disregard the request .
8) the agent will wait for 60 seconds and as it did not receive any response ,it will throw the "SiteMinder Agent Api function failed - 'Sm_AgentApi_IsProtectedEx' returned '-2' which means that the request was timed out followed by the -1 as the Agent will get in into the re-connect mode
To solve this issue; the idle timeout configured on the policy server should be less than the session timeout configure for any device between policy server and agent (load balancer or firewall)
Additional Information
docops.ca.com for your version for updated information
Feedback
