search cancel

JBoss agent unable to handshake with policy server while trusted host registration is successful.


Article ID: 36080


Updated On:


CA Single Sign On Secure Proxy Server (SiteMinder) CA Single Sign On SOA Security Manager (SiteMinder) CA Single Sign-On



After trusted host registration (either via wizard or smreghost) from JBoss agent to Policy server was successful, trusted host was generated at Policy server WAMUI.

However, when you startup JBoss agent, JBoss server log complains Shared Secret invalid and smps.log reported handshake error

 Snippet from JBoss server.log:
 12:06:18,740 INFO  [stdout] (http-/ 12:06:18.740 [http-/] SMTRACE: SmAgentTliSession, setup, Initiating TLI handshake
 12:06:18,740 INFO  [stdout] (http-/ 12:06:18.740 [http-/] SMTRACE: SmConfigAttribute, decrypt, Attempting to decrypt input = {RC2}v90t13apwY+eh74vWZZC2l+d1bXRHAeharsqUfRLKjoy/pboPD8WU+kMLENW6
 12:06:18,740 ERROR [stderr] (http-/ 12:06:18.740 [http-/] SMERROR: SmServerConnection, handshake, Failed session setup.
 12:06:18,755 ERROR [stderr] (http-/ Shared secret invalid.
 12:06:18,755 ERROR [stderr] (http-/      at*..:241)
 12:06:18,755 ERROR [stderr] (http-/      at*..:324)
 12:06:18,771 ERROR [stderr] (http-/      at*..:409)
 12:06:18,771 ERROR [stderr] (http-/      at*..:304)
 12:06:18,771 ERROR [stderr] (http-/      at*..:235)
 12:06:18,787 ERROR [stderr] (http-/      at*..:646)
 12:06:18,787 ERROR [stderr] (http-/      at*..:216)
 12:06:18,787 ERROR [stderr] (http-/      at*..:114)
 12:06:18,802 ERROR [stderr] (http-/      at*..:128)
 12:06:18,802 ERROR [stderr] (http-/      at netegrity.siteminder.javaagent.AgentAPI.a(DashoA10*..:934)
 12:06:18,802 ERROR [stderr] (http-/      at netegrity.siteminder.javaagent.AgentAPI.getConfig(DashoA10*..:1256)
 12:06:18,802 ERROR [stderr] (http-/      at
 12:06:18,818 ERROR [stderr] (http-/      at<init>(
 12:06:18,818 ERROR [stderr] (http-/      at
 12:06:18,818 ERROR [stderr] (http-/      at
 12:06:18,833 ERROR [stderr] (http-/      at
 12:06:18,833 ERROR [stderr] (http-/      at<clinit>(
 12:06:18,833 ERROR [stderr] (http-/      at
 12:06:18,849 ERROR [stderr] (http-/      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
 12:06:18,849 ERROR [stderr] (http-/      at
 12:06:18,849 ERROR [stderr] (http-/      at org.apache.catalina.core.StandardHostValve.invoke(
 12:06:18,849 ERROR [stderr] (http-/      at org.apache.catalina.valves.ErrorReportValve.invoke(
 12:06:18,849 ERROR [stderr] (http-/      at org.apache.catalina.core.StandardEngineValve.invoke(
 12:06:18,865 ERROR [stderr] (http-/      at org.apache.catalina.connector.CoyoteAdapter.service(
 12:06:18,865 ERROR [stderr] (http-/      at org.apache.coyote.http11.Http11Processor.process(
 12:06:18,865 ERROR [stderr] (http-/      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(
 12:06:18,865 ERROR [stderr] (http-/      at$
 12:06:18,880 ERROR [stderr] (http-/      at
 12:06:18,880 ERROR [stderr] (http-/ Caused by: No such algorithm: RC2/CBC/PKCS5Padding
 12:06:18,880 ERROR [stderr] (http-/      at<init>(DashoA10*..:76)

Snippet from smps.log
 [CServer.cpp:1971][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3152
 [CServer.cpp:1978][ERROR][sm-Tunnel-00030] Handshake error: Failed to receive client hello. Socket error 0
 [CServer.cpp:2143][ERROR][sm-Server-01070] Failed handshake with


 Communication between JBoss agent and policy server relied on Java Cryptography Extension (JCE).
 The hints in the server.log error is SmCryptoProviderException.

 1. First thing to check is whether JCE is patched to the Java used by JBoss.
 2. If JCE is patched but issue persists, this could be due to the JBoss agent point to incorrect Java.

 During JBoss agent installation, the installer asked to choose Java Virtual Machine.

<Please see attached file for image>


The installer asked to choose 32-bit JRE but that’s incorrect. This is the installer defect that CA is aware and will be addressed in future release. What Java to choose need to base on what JBoss server used. If JBoss server is 64-bit, we need to choose 64-bit Java

<Please see attached file for image>



<Please see attached file for image>


In other word, if you have selected wrong java during JBoss agent installation, it will cause agent failed to communicate with policy server even though trusted host registration was successful.

Please try to uninstall JBoss agent and reinstall then choose the correct Java to use.

Additional Information:
JBoss agent: R12.52SP1
JBoss server: 6.4
JBoss server OS: Windows 2008R2


Component: SMJBSS


1558722584720000036080_sktwi1f5rjvs16wic.png get_app
1558722582768000036080_sktwi1f5rjvs16wib.png get_app
1558722580611000036080_sktwi1f5rjvs16wia.png get_app