Can't connect to CA DIRECTORY policystore

Products

• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• CA Single Sign-On
• SITEMINDER

Issue/Introduction

Error 81 after switching LDAP Policy store from Oracle to CA directory for SSL

Customer test the connection with JExplorer over SSL and using Policy server tool LDAPSearch 

/opt/siteminder/bin/ldapsearch -b "o=siteminder" -h example.com:636 -Z -P /opt/siteminder/certdb/cert7.db -D "cn=<name>,o=example.com,c=us" -w <passwd>  cn=* 2>&1                                   

Both test successful 

Environment

Linux 6 policy Policy server connecting to CA directory 12.0 SP14 on liniux over SSLInformation regarding SSO policy server and communication to LDAP servers

Cause

Steps performed that failed:

1. Opened smconsole, and clicked apply with the current settings, connect over ssl is always successful here 
2. Changed the connect strings and cert7 location or cert8.db if newer version of SSO, clicked apply, then clicked test connection...this Failure with LDAP Error 81 can't contact ldap error

Troubleshooting steps 

1. Validate/modify certx.db files with the right PEM from CA dir – use the information above to determine which NSS kit to use (NSS can be downloaded mozzilla web site) 
2. PEM file location <CA_Dir_HOME>dxserver/config/ssld  (trusted.pem) folder /personalities the directory instance PEM (example below example-6668.pem)

Resolution

Proper steps to test/Change from Oracle LDAP to CA Directory over SSL 

1. Opened SMCONSOLE, and clicked apply with the current settings, connect over ssl is always successful here 
2. Changed the connect strings and cert7 location or cert8.db if newer version of SSO, clicked apply
3. Exit SMCONSOLE – this is needed to update semaphores on Linux
4. Reopen SMCONSOLE clicked test connection...this successful; At this point you can stop and start policy server process to load the store form is new destination 

Additional Information

The Policy Server uses a Mozilla LDAP SDK to communicate with LDAP directories (Policy store/User Store etc.).  The Mozilla LDAP SDK implements the NSS kit/libraries.  Support for security protocols SSL/TLS 1.0/1.1/1.2 … depends on the bundled NSS libraries used by the specific policy server 

• R12.SP3CR12 and below = NSS 3.3.2.0 – Only SSL protocol
• R12.51CR6 onwards = NSS 3.14.3.0  - TLS v 1.1
• R12.52SP1 CR1 onwards = NSS 3.14.3.0 – TLS v 1.1
• R12.52SP2 until CR1  = NSS 3.14.3.0 0 – TLS v 1..1
• R12.6 = NSS 3.20 – TLS v 1.2

How to create/add PEM files the DB file:

• C:\nss-3.3.2\db>certutil -A -n My-rootca -t "C,," -i trusted.pem -d .
• C:\nss-3.3.2\db>certutil -A -n My-6668 -t "P,," -i example-6668.pem -d .

Copy all DB files (cert7.db of cert8.db, key3.db, secmod.db) to the location defined in smconsole

SMCONSOLE (Netscape Certificate Database file – pints to the cert7.db/cert8.db)