Server Error. The server was unable to process your request or Communication failure between SiteMinder policy server and web agent

Products

• SITEMINDER
• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On Agents (SiteMinder)

Issue/Introduction

SiteMinder Access Gateway Users randomly receive "Server Error. The server was unable to process your request" in their browsers.

SiteMinder agent and agent trace logs display:

[CSmLowLevelAgent.cpp:595][ERROR][sm-AgentFramework-00520] LLA: SiteMinder Agent Api function failed - 'Sm_AgentApi_IsProtectedEx' returned '-1'.
[CSmProtectionManager.cpp:192][ERROR][sm-AgentFramework-00420] HLA: Component reported fatal error: 'Low Level Agent'.
[CSmHighLevelAgent.cpp:417][ERROR][sm-AgentFramework-00420] HLA: Component reported fatal error: 'Protection Manager'.

...

[CSmLowLevelAgent.cpp:590][IsResourceProtected][][*x.x.x.x][][example-agent][][][][GET][][][][][][][][][][][Communication failure between SiteMinder policy server and web agent.]

However, many web servers in the same data center communicate with the same policy servers as the access gateway. No other server is experiencing the same error.

Environment

Release : 12.8.05

Cause

Sometimes "Communication failure between SiteMinder policy server and web agent" is caused by network interruption.

However, in this case, it is not due to network.  

During the agent trace log analysis, notice that SMAGENTNAME does not have proper leading smencoding, e.g. -SM-, or $SM$.
When this happens, the policy server usually can not find the matching agent name, then it will result to above error. Which is expected by design.

[09/06/2022][20:15:38.484][2192][2744][CSmHighLevelAgent.cpp:399][ProcessRequest][][*x.x.x.x][][example-agent][/siteminderagent/ntlm/creds.ntc?CHALLENGE=&SMAGENTNAME=qwsJPfnc............................LiLE&TARGET=-SM-https%253a%252f%252fwww%252e....aspx][][][GET][][][][][][][][][][][ProtectionManager returned SmNo, end new request.]

[09/06/2022][20:15:38.484][2192][2744][CSmLowLevelAgent.cpp:590][IsResourceProtected][][*x.x.x.x][][example-agent]][][][][GET][][][][][][][][][][][Communication failure between SiteMinder policy server and web agent.]

On the policy server side trace log, one may find matching error of the same request like below, because the SMAGENTNAME was passed with incorrect value:

[ERROR][sm-IsAuthorized-00220] Bad sxxx/rxxx request detected: error 'Cannot fetch agent xxxx'

The assumption here is that an end user hits a bookmarked url already with wrong value. If this application is in a public facing domain, then there is very limited control over end user's browser settings or what kind of request a user sends.

Resolution

• The root cause of the error "Communication failure between SiteMinder policy server and web agent" is due to wrong SMAGENTNAME was submitted during URL request.

In the failed request, SMAGENTNAME does not have proper leading smencoding, e.g. -SM-, or $SM$.

When this happens, the policy server usually can not find the matching agent name from policy store after it completes its exhaustive trusted host search, then it will result to above error.

If the bad request is coming from a legit user, then ask end user to use properly formatted SMAGENTNAME or correct old bookmarked url and application program used.

If this client ip is NOT legit, then block it from reaching web agent at network security or load balancer level.

• There is another possible scenario, where even if it had the correct smencoding, but the agent name is not actually in the trusted host list from the policy store, then it will reach the same result.

Typically when data center is upgraded/migrated, new policy servers were introduced. And old agents keep sending requests to new data center but forgot to be registered with the new data center policy servers.

Additional Information

https://knowledge.broadcom.com/external/article?articleId=39387