Getting multiple alerts on Web Agent server

book

Article ID: 141422

calendar_today

Updated On:

Products

CA Single Sign On Secure Proxy Server (SiteMinder) CA Single Sign On SOA Security Manager (SiteMinder) SITEMINDER

Issue/Introduction

 

We're running a Web Agent and when the user tries to access

 https://myserver.mydomain.com/myweb/mypage.jsp

then the browser receives return code 500 with the mention :

    Internal Server Error

    The server encountered an internal error or misconfiguration and was
    unable to complete your request.

    Please contact the server administrator at [email protected] to
    inform them of the time this error occurred, and the actions you
    performed just before this error.

    More information about this error may be available in the server error
    log.

    Additionally, a 500 Internal Server Error error was encountered while
    trying to use an Error Document to handle the request.

How can we fix this ?

 

Cause

 

Applying the following template to the Web Agent traces config file :

components: AgentFramework, HTTPAgent, WebAgent, AgentFunc, Agent_Con_Manager
data: Date, Time, PreciseTime, Pid, Tid, TransactionID, Message, SrcFile, Function, RequestIPAddr, IPAddr, IPPort, AgentName, Domain, DomainOID, Realm, RealmOID, Resource, Action, User, SessionSpec, SessionID, CertSerial, SubjectDN, IssuerDN, UserDN, Threshold, Throughput, MinThroughput, MaxThroughput, HandleCount, BusyHandleCount, FreeHandleCount, State, ClusterID

we can see that the Web Agent thread lost connection with the Policy
Server as it waits too long to get the response back at the network
level.

WebAgent.log

  [21557/2734651136][Thu Nov 28 2019
  08:25:31][SmAgentAPI.cpp:315][INFO][sm-AgentConMgr-00050] Failover
  from cluster [0] to cluster [1].

[...]

  [21557/2766120704][Thu Nov 28 2019
  08:26:38][CSmLowLevelAgent.cpp:546][ERROR][sm-AgentFramework-00520]
  LLA: SiteMinder Agent Api function failed -
  'Sm_AgentApi_IsProtectedEx' returned '-2'.

[...]

  [21557/2703181568][Thu Nov 28 2019
  08:26:57][CSmLowLevelAgent.cpp:546][ERROR][sm-AgentFramework-00520]
  LLA: SiteMinder Agent Api function failed -
  'Sm_AgentApi_IsProtectedEx' returned '-1'.

WebAgentTrace.log 

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][]["SmAgentT
  li_TCPReceiveBuf returns TIME-OUT. No connection retry."][SmClient.
  cpp:1673][CSmServerHandle::ProcessRequest][][10.1.1.0][44443][
  ][][][][][][][][][][][][][][][][][][3][][][ACTIVE][]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Request f
  ailed with 2 error code.][SmClient.cpp:2745][ProcessRequest][][][][
  ][][][][][][][][][][][][][][][][][][4][][][ACTIVE][9]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Server St
  ate transition from ACTIVE to INTER][SmClient.cpp:1810][CSmServerHa
  ndle::StateTransition][][10.0.0.1][44443][][][][][][][][][][][
  ][][][][][][][][3][][][INTER][]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Request f
  ailed with 2 error code.][SmClient.cpp:2762][ProcessRequest][][][][
  ][][][][][][][][][][][][][][][][][][4][][][ACTIVE][9]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Server St
  ate transition from INTER to INACTIVE][SmClient.cpp:1810][CSmServer
  Handle::StateTransition][][10.0.0.1][44443][][][][][][][][][][
  ][][][][][][][][][3][][][INACTIVE][]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][The numbe
  r of active servers fell below the threshold.][SmClient.cpp:2846][P
  rocessRequest][][][][][][][][][][][][][][][][][][][][][][4][][][ACT
  IVE][9]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Cluster S
  tate transition from ACTIVE to INACTIVE][SmClient.cpp:3273][CSmClus
  ter::StateTransition][][][][][][][][][][][][][][][][][][][][][][4][
  ][][INACTIVE][9]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][No active
  clusters found.][SmAgentAPI.cpp:521][ProcessRequest][][][][][][][]
  [][][][][][][][][][][][][][][][][][][]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Leave fun
  ction ProcessRequest][SmAgentAPI.cpp:527][ProcessRequest][][][][][]
  [][][][][][][][][][][][][][][][][][][][][]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][][Leave fun
  ction Sm_AgentApi_IsProtected][SmAgentAPI.cpp:1813][Sm_AgentApi_IsP
  rotected][][][][][][][][][][][][][][][][][][][][][][][][][][]

  [11/28/2019][08:26:38][08:26:38.830][21557][2766120704][00000000000
  000000000000064f45899-5435-5ddf7672-a4dfa700-2cd73432c50][Communica
  tion failure between SiteMinder policy server and web agent.][CSmLo
  wLevelAgent.cpp:541][IsResourceProtected][10.0.0.2][*10.0.0.3][]
  [myagent][][][][][/myweb/mypage.jsp][GET][DOMAIN\MYUSER]
  [k3ZVtGzXIMnuIZ1qHzR7BxzYMbzvuwL8PLJH

  [...]

  qdpI=][][][][CN=myname,DC=mydomain,DC=com][][][][][][][][][]

 

Environment

 

Release : 12.7

Component : SITEMINDER -POLICY SERVER

 

Resolution

 

- Investigate network, load balancer and firewall and make sure that
  there's no timeout on the TCP Protocol. If there's one, make sure
  that the TCP Protocol timeout is big enough.

- On the Web Agent and Policy Server, enable the environment variable
  SM_ENABLE_TCP_KEEPALIVE to insure that both component won't try to
  use a connection that has been terminated on the firewall or
  loadbalancer :

  SM_ENABLE_TCP_KEEPALIVE 

  Web Agent and Policy Server Network Communication Disruption
  https://ca-broadcom.wolkenservicedesk.com/external/article?articleId=42108