Java Agent API cannot connect to policy server

Article Id: 102858
Status: Published
Updated On: 20-06-2018 11:45
Products:
CA Single Sign On Secure Proxy Server (SiteMinder) , AXIOMATICS POLICY SERVER , CA Single Sign On SOA Security Manager (SiteMinder) , CA Single Sign-On , CA Single Sign-On
Issue/Introduction:

When I port my working Java Agent code from R12 to R12.52, it generate the following error on R12.52 policy server side:

[65180/4086831984][Mon Jun 04 2018 13:30:52][CServer.cpp:2016][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3152 [65180/4086831984][Mon Jun 04 2018 13:30:52][CServer.cpp:2023][ERROR][sm-Tunnel-00030] Handshake error: Failed to receive client hello. Socket error 104
[65180/4086831984][Mon Jun 04 2018 13:30:52][CServer.cpp:2188][ERROR][sm-Server-01070] Failed handshake with 10.150.138.171:53448

Client side log clip:
====================>>>
17:46:05.410 [main] SMTRACE: SmCluster, enable, Attempting to enable cluster id = 3
17:46:05.410 [main] SMTRACE: SmCluster, enable, Attempting to enable server index = 0
17:46:05.410 [main] SMTRACE: SmServer, enable, started
17:46:05.410 [main] SMTRACE: SmServer, createConnections, Attempt to create connections = 2
17:46:05.410 [main] SMINFO: SmServer, retryTimedoutConnections, Error retrying connection NO CONN
17:46:05.410 [main] SMTRACE: SmServer, createConnection, Currently have 0 connections to server.
17:46:05.410 [main] SMINFO: SmServer, createConnection, Creating a server connection, index = 0
17:46:05.410 [main] SMTRACE: SmServerConnection, init, Attempt to init connection
17:46:05.410 [main] SMTRACE: SmAgentTcpTransport, newInstance, Using SmAgentTcpTransport class
17:46:08.411 [main] SMTRACE: SmAgentTliSession, setup, Initiating TLI handshake
17:46:08.411 [main] SMTRACE: SmConfigAttribute, decrypt, Attempting to decrypt input = {RC2}lv5HPIIpFRvODEg+VU1ocFfnI4jzwmFPtQ5B87lOaW5dgaX+nFZ+JYyRs0MaBskqFKBet6bWi326pbmm8s0nkCTsb+kNRvDvQFcDoNcBEKJz+qah47KrLuJ6S2ak8DO83t1mJ/vG8j9QfbxqbrDaxU5wybJP7NrGysgRuGXOtRnLbOTkN/EERHc20rgy4enM
17:46:08.411 [main] SMTRACE: SmCryptoProvider, init, Setting keys
17:46:08.411 [main] SMTRACE: SmCryptoProvider, init, Setting IV
17:46:08.411 [main] SMTRACE: SmCryptoProvider, init, Initializing params
17:46:08.411 [main] SMTRACE: SmCryptoProvider, init, Init complete
17:46:08.411 [main] SMTRACE: SmCryptoProvider, decrypt, Using crypto provider = com.ca.siteminder.sdk.agentapi.crypto.SmRC2SHA1CryptoProvider, on input length = 192
17:46:08.411 [main] SMTRACE: SmCryptoProvider, decrypt, Getting decrypter
17:46:08.412 [main] SMTRACE: SmCryptoProvider, decrypt, Got decrypter
17:46:08.413 [main] SMINFO: SmServer, releaseConnection, Released connection 34a97744, count = 0
17:46:08.413 [main] SMINFO: SmCluster, enable, Failed to activate cluster id = 3
2018-06-04 17:46:08:414 AgentApi.getConfig() returns  FAILURE (-1)
Connection try 4: Fail to call AgentApi.getConfig()
FATAL: Fail to call AgentAPI.getConfig() for xyz159 after 4 tries.
<<<=================================
 

Cause:

 The "<sdk_install_dir>/bin/smreghost" created SmHost.conf with contents for JNI based agent.

Environment:

Policy Server version: 12.52.105.2112 on
RH Linux: Linux dvlva177 2.6.32-696.18.7.el6.x86_64 #1 SMP Thu Dec 28 20:15:47 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
SDK version: 12.52.107.2259

Resolution:

Need to run:
"<sdk_install_dir]/bin/smreghost.sh>" for Pure Java agent to communicate with  the policy server over TLI protocol..