Load Balancer handshake errors in smps log

Products

• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On Agents (SiteMinder)
• CA Single Sign On Federation (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• SITEMINDER

Issue/Introduction

We are implementing SiteMinder R12.8.03 and we are using F5 load balancer between Webagent and Policy server. Though the siteminder core functionalities are working all good, but we are getting a lot of handshake errors for Load balancer IP in smps log. It is difficult to check for any SiteMinder related messages in smps log file as quite a huge amount of error messages for Load Balancer handshake errors.
We want to know how to configure the Load balancer so that we can stop the load balancer handshake erros in smps log file.

[28313/140068790454016][Mon Aug 17 2020 22:15:33][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3159

[28313/140068790454016][Mon Aug 17 2020 22:15:33][CServer.cpp:2126][ERROR][sm-Tunnel-00020] Handshake error: Failed to receive client hello. Client disconnected

Environment

Release : 12.8.03

Component : SITEMINDER -WEB AGENT FOR APACHE

Resolution

Messages in logs were removed by setting load balancer health check to use TCP Half Open:

https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/layer7-identity-and-access-management/single-sign-on/12-52-01/configuring/configure-agent-to-policy-server-communication-using-a-hardware-load-balancer.html#concept.dita_48ba0e488c419779764669189b189f442aa6ecc2_MonitoringtheHealthofHardwareLoadBalancingConfigurations