Error : Agent Api function failed with Web Agent and Load Balancer

Products

CA Single Sign On Secure Proxy Server (SiteMinder) CA Single Sign On SOA Security Manager (SiteMinder) CA Single Sign-On SITEMINDER CA Single Sign On Agents (SiteMinder)

Issue/Introduction

Introducing a Load Balancer between the Web Agent and the Policy
server can cause Commutation failure errors that can be identified in
the Web Agent Log starting with -2 followed by -1 Error as seen
below :

  [14259/1151969248][Sun Feb 07 2016 12:58:21][CSmLowLevelAgent.cpp:546][ERROR]
  [sm-AgentFramework-00520]
  LLA:SiteMinder Agent Api function failed - 'Sm_AgentApi_IsProtectedEx' returned '-2'.

  [14257/1151969248][Sun Feb 07 2016 13:06:58][CSmProtectionManager.cpp:192][ERROR]
  [sm-AgentFramework-00420]HLA: Component reported fatal error: 'Low Level Agent'.

  [14257/1151969248][Sun Feb 07 2016 13:06:58][CSmHighLevelAgent.cpp:413][ERROR]
  [sm-AgentFramework-00420] HLA:Component reported fatal error: 'Protection Manager'.

  [14257/1151969248][Sun Feb 07 2016 13:07:55][CSmLowLevelAgent.cpp:1378][ERROR]
  [sm-AgentFramework-00520]
  LLA:SiteMinder Agent Api function failed - 'Sm_AgentApi_LoginEx' returned'-1'.

  [14257/1151969248][Sun Feb 07 2016 13:17:10][CSmLowLevelAgent.cpp:1378][ERROR]
  [sm-AgentFramework-00520]
  LLA:SiteMinder Agent Api function failed - 'Sm_AgentApi_LoginEx' returned'-1'.

  [14257/1151969248][Sun Feb 07 2016 13:17:10][CSmAuthenticationManager.cpp:194]
  [ERROR][sm-AgentFramework-00420] HLA: Component reported fatal error: 'Low Level Agent'.

  [14257/1151969248][Sun Feb 07 2016 13:17:10][CSmHighLevelAgent.cpp:1244][ERROR]
  [sm-AgentFramework-00420] HLA: Component reported fatal error: 'Authentication Manager'

Cause

As a Load Balancer has been introduced between the Web Agent and the
Policy server, here is what it can happen :

  - The Policy server has a TCP Idle Timeout value set to 10 min
    (default) which is configurable from the smconsole. What that
    means is that the Policy Server will timeout an active connection
    from Web Agent if no request received on it within 10 min;

  - When a Load Balancer is introduced in the middle, most of these
    Load Balancers has the session timeout set to 5 min hence the
    problem;

The steps below explains how the problem occurs

  1) Web Agent opens a connection to the Policy Server where Normal
     priority requests are served;
     
  2) If the Web Agent doesn't send any request on this connection
     within 5 minutes, the Load Balancer will timeout the session and
     close the connection;
     
  3) The problem arises as the Load Balancer doesn't notify the Web
     Agent nor the Policy Server of this closed connection;
     
  4) For the Policy Server, it will terminate the connection from its
     end after another 5 minutes (total 10 minutes) as no connection
     was received from Web Agent within the last 10 minutes. The
     Policy Server will notify the Web Agent that connection was
     closed;
     
  5) The problem is that the Load Balancer receives the close
     connection from Policy Server, however, as the Load Balancer
     already dropped the connection as explained in Step 3, the close
     request never reaches the Web Agent;

  6) Now the Web Agent receives requests that he needs to process, it
     looks for the available connections in the pool and as the
     connection in question is still available, the Web Agent will
     process the call and by default will wait for 60 seconds to hear
     back from the Policy Server;

  7) The Load Balancer receives the request from Web Agent and as it
     dropped already the connection, it will disregard the request;
     
  8) The Web Agent will wait for 60 seconds and as it did not receive
     any response, it will throw the

       "SiteMinder Agent Api function failed - 'Sm_AgentApi_IsProtectedEx' returned '-2' 

     which means that the request was timed out followed by the -1 as
     the Web Agent will get in into the re-connect mode;

Resolution


- To solve this issue, the idle timeout configured on the Policy
  Server should be less than the session timeout configure for any
  device between Policy Server and Web Agent (Load Balancer or
  Firewall) (1).

Additional Information

(1)

    Idle Timeouts and Stateful Inspection Devices

      Stateful inspection devices, such as firewalls, generally have an
      idle timeout setting. SiteMinder connections from Policy Servers to
      Agents also have idle timeout settings.

    https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/siteminder/12-8/troubleshooting/policy-server-troubleshooting.html