Introduction:
After trusted host registration (either via wizard or smreghost) from JBoss agent to Policy server was successful, trusted host was generated at Policy server WAMUI.
However, when you startup JBoss agent, JBoss server log complains Shared Secret invalid and smps.log reported handshake error
Snippet from JBoss server.log:
12:06:18,740 INFO [stdout] (http-/0.0.0.0:8080-1) 12:06:18.740 [http-/0.0.0.0:8080-1] SMTRACE: SmAgentTliSession, setup, Initiating TLI handshake
12:06:18,740 INFO [stdout] (http-/0.0.0.0:8080-1) 12:06:18.740 [http-/0.0.0.0:8080-1] SMTRACE: SmConfigAttribute, decrypt, Attempting to decrypt input = {RC2}v90t13apwY+eh74vWZZC2l+d1bXRHAeharsqUfRLKjoy/pboPD8WU+kMLENW6
12:06:18,740 ERROR [stderr] (http-/0.0.0.0:8080-1) 12:06:18.740 [http-/0.0.0.0:8080-1] SMERROR: SmServerConnection, handshake, Failed session setup.
12:06:18,755 ERROR [stderr] (http-/0.0.0.0:8080-1) com.ca.siteminder.sdk.agentapi.tli.a5: Shared secret invalid.
12:06:18,755 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.tli.a7.a(DashoA10*..:241)
12:06:18,755 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.v.h(DashoA10*..:324)
12:06:18,771 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.s.c(DashoA10*..:409)
12:06:18,771 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.s.a(DashoA10*..:304)
12:06:18,771 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.s.d(DashoA10*..:235)
12:06:18,787 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.o.d(DashoA10*..:646)
12:06:18,787 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.a3.a(DashoA10*..:216)
12:06:18,787 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.connection.a3.a(DashoA10*..:114)
12:06:18,802 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.w.a(DashoA10*..:128)
12:06:18,802 ERROR [stderr] (http-/0.0.0.0:8080-1) at netegrity.siteminder.javaagent.AgentAPI.a(DashoA10*..:934)
12:06:18,802 ERROR [stderr] (http-/0.0.0.0:8080-1) at netegrity.siteminder.javaagent.AgentAPI.getConfig(DashoA10*..:1256)
12:06:18,802 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.core.EvaluatorFactory.connectToPolicyServer(EvaluatorFactory.java:982)
12:06:18,818 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.core.EvaluatorFactory.<init>(EvaluatorFactory.java:254)
12:06:18,818 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.core.EvaluatorFactory.getInstance(EvaluatorFactory.java:140)
12:06:18,818 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.core.EvaluatorFactory.getInstance(EvaluatorFactory.java:112)
12:06:18,833 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.core.SMAgentInitializer.init(SMAgentInitializer.java:71)
12:06:18,833 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.core.SMAgentInitializer.<clinit>(SMAgentInitializer.java:20)
12:06:18,833 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.soa.agent.appserver.authenticator.jboss.SMJBoss6BasicAuthenticator.authenticate(SMJBoss6BasicAuthenticator.java:39)
12:06:18,849 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
12:06:18,849 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
12:06:18,849 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150)
12:06:18,849 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
12:06:18,849 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
12:06:18,865 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
12:06:18,865 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854)
12:06:18,865 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653)
12:06:18,865 ERROR [stderr] (http-/0.0.0.0:8080-1) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926)
12:06:18,880 ERROR [stderr] (http-/0.0.0.0:8080-1) at java.lang.Thread.run(Thread.java:745)
12:06:18,880 ERROR [stderr] (http-/0.0.0.0:8080-1) Caused by: com.ca.siteminder.sdk.agentapi.crypto.SmCryptoProviderException: java.security.NoSuchAlgorithmException: No such algorithm: RC2/CBC/PKCS5Padding
12:06:18,880 ERROR [stderr] (http-/0.0.0.0:8080-1) at com.ca.siteminder.sdk.agentapi.crypto.ar.<init>(DashoA10*..:76)
Snippet from smps.log
[CServer.cpp:1971][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3152
[CServer.cpp:1978][ERROR][sm-Tunnel-00030] Handshake error: Failed to receive client hello. Socket error 0
[CServer.cpp:2143][ERROR][sm-Server-01070] Failed handshake with 127.0.0.1:56499
Instructions:
Communication between JBoss agent and policy server relied on Java Cryptography Extension (JCE).
The hints in the server.log error is SmCryptoProviderException.
1. First thing to check is whether JCE is patched to the Java used by JBoss.
2. If JCE is patched but issue persists, this could be due to the JBoss agent point to incorrect Java.
During JBoss agent installation, the installer asked to choose Java Virtual Machine.
<Please see attached file for image>
<Please see attached file for image>
<Please see attached file for image>
Additional Information:
JBoss agent: R12.52SP1
JBoss server: 6.4
JBoss server OS: Windows 2008R2