Troubleshooting Siteminder Application Server Agent (ASA) 6.0 Host Registration.

Products

CA Single Sign On Secure Proxy Server (SiteMinder) CA Single Sign On SOA Security Manager (SiteMinder) CA Single Sign-On

Issue/Introduction

Description:

Upon attempting to register the Siteminder ASA with the Policy Server (trusted host registration), the error, "Failed to enable any clusters." is thrown, either as console output after running 'smreghost', or while running the configuration script. At the Policy Server side, "Bad security handshake attempt." errors are seen along with this error.

**NOTE: You must use the 'smreghost' program packaged with the Siteminder ASA, the 'SmHost.conf' generated from a standard Web Agent's 'smreghost' is incompatible with the ASA.

Solution:

PROBLEM: The Policy Server may be unreachable, due to either a network failure event or a firewall misconfiguration.
SOLUTION: To confirm this, use the "telnet" application on the ASA machine, and attempt to establish connections to all three Policy Server ports (by default ports 44441,44442,44443, but check your Policy Server configuration to be sure). If the connection is immediately refused or you receive a timeout message, then the next step is to engage your network administrators to resolve the underlying network issue. If this is the reason for failure, you will not see any errors in the Policy Server error log, as the request was never actually received by the Policy Server.
PROBLEM: Can be caused by using a Java version that has not been patched with the Java unlimited-strength encryption (JCE) patch.
SOLUTION: Determine which version of Java is being used (e.g. check the system path to determine which Java binary is actually executed from the command line) and apply the JCE patch per the documentation provided by the vendor of the JCE patch (either Sun or IBM support). Ensure the JAVA_HOME environment variable is also set on unix installations within the 'smreghost.sh' script. (NOTE: the JAVA_HOME variable does not necessarily determine the java version used; the system path determines which java binary is run.)
PROBLEM: In Unix, the Siteminder admin username and/or password contains special characters or spaces, that are not properly parsed.
SOLUTION: Use double-quotes around admin username and password.
PROBLEM: In Unix, the HCO name contains spaces.
SOLUTION: To use an HCO name containing spaces, you must enclose it in quotes on the commandline. If you still see an error after trying this, there is a known issue with earlier versions of the Siteminder ASA v6.0 for WebSphere that you may be seeing, where even using quotes may not allow the HCO name to parse. This particular issue was fixed in the Siteminder ASA v6.0-CR07 for WebSphere release (see hotfix #64451).
PROBLEM: The Siteminder admin account being used is an external account (e.g. the Siteminder administrator resides in an external user directory, rather than locally in the Policy Store)
SOLUTION: This problem was identified as a bug in the Pure Java Agent API, and has been fixed in the ASA v6.0 CR-008 version (see hotfix #67693).
PROBLEM: If you are using an IBM version of Java, but applied the Sun version of the Unlimited-Strength Encryption (JCE) patch, you will see this error upon running 'smreghost.sh'.
-------------
ERROR:
"java.lang.SecurityException: Cannot set up certs for trusted CAs"
"java.lang.SecurityException: Jurisdiction policy files are not signed by trusted signers!"
-------------
SOLUTION: Replace the incorrect JCE patch with the correct vendor's version. If you are using the IBM JDK, please contact IBM Support (http://www.ibm.com/support/). If using the Sun JDK, contact Sun Support (http://java.sun.com/).

Environment

Release:
Component: SMBEA