Getting handshake errors in SMPS logs

Products

• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• CA Single Sign-On

Issue/Introduction

We're running Policy Servers, and suddenly we see them starting
experiencing High CPU consumption. This results in Web Agents were
unable to connect to the Policy Server.

We've changed nothing to Siteminder environment.

How can we solve that ?

Environment

Policy Server 12.52SP1CR02

Cause

You have taken some pstack from the Policy Server process at the time
of the issue.

The stacks show a known issue occuring in the Policy Server 12.52SP1CR02. 

This issue occurs when one or more users belong to circular groups in
your User Directory.

Let's say : 

user1 is in group 1, which group belongs to group 4, group 4 to group 
5, and group 5 belongs to group 1 : This would be circular group ... 

Previous to Policy Server 12.52SP1CR04 this kind of User Directory 
configuration brought the Policy Server to consume all the available 
CPU cycles. 

The stack reported from your environment match the pstack that have 
been analysed in the fix : 

Defects Fixed in 12.52 SP1 CR04 

High CPU Usage 
Policy Server experiences high CPU usage. 

STAR Issue: 00180246 

RTC Issue: 163452/DE112391 

https://docops.ca.com/ca-single-sign-on/12-52-sp1/en/release-notes/cumulative-releases/defects-fixed-in-12-52-sp1-cr04 

This fix ticket reported the culprit threads show the following lines : 

icuin49!icu_49::RuleBasedCollator::compare+0x34 
icuin49!icu_49::Collator::compareUTF8+0x87 
smi18n!SmCompareUTF8+0x9c 
smi18n!SmUTF8StrICompare+0x18 
smdsldap!CSmDsLdapProvider::HasRelationship+0x63c 

As the pstack from your environment show the same stack, then it has
the same.
 

Resolution

Upgrade the Policy Server to at least version 12.52SP1CR04 to solve
this issue.