problems with SSL connection for SDM connecting to Maileater, PAM, or Catalog
search cancel

problems with SSL connection for SDM connecting to Maileater, PAM, or Catalog

book

Article ID: 103456

calendar_today

Updated On:

Products

CA Service Management - Service Desk Manager CA Service Desk Manager CA Service Catalog CA Process Automation Base

Issue/Introduction

After either upgrading to Service Desk Manager, or configuring a new install with SSL for the first time, SSL doesn't work.

In some cases an OAuth token may be generated, but Emails are not processed

In the maileater_nxd.log (debug mode) you may see messages similar to this:  (specific entries of interest highlighted)

[Thread-3] c.c.S.m.c.PDMMailerUtil - [pdm_perl, pdm_keystore_mgr.pl, -import, C:\certs\cert.cer] 
[Thread-5] c.c.S.m.c.PDMMailerUtil - keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect 
[Thread-5] c.c.S.m.c.PDMMailerUtil - java.io.IOException: Keystore was tampered with, or password was incorrect 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.provider.JavaKeyStore.engineLoad(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.provider.JavaKeyStore$JKS.engineLoad(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.provider.KeyStoreDelegator.engineLoad(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at java.security.KeyStore.load(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.tools.keytool.Main.doCommands(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.tools.keytool.Main.run(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - at sun.security.tools.keytool.Main.main(Unknown Source) 
[Thread-5] c.c.S.m.c.PDMMailerUtil - Caused by: java.security.UnrecoverableKeyException: Password verification failed 
[Thread-5] c.c.S.m.c.PDMMailerUtil - ... 8 more 
[Thread-4] c.c.S.m.c.PDMMailerUtil - 
DEBUG [Thread-4] c.c.S.m.c.PDMMailerUtil - FAILED: The certificate was not imported into the keystore. 
DEBUG [Thread-4] c.c.S.m.c.PDMMailerUtil - Exiting at pdm_keystore_mgr.pl line 170. 
DEBUG [Thread-3] c.c.S.m.c.PDMMailerUtil - Exit value from pdm_keystore_mgr.pl: 1 

You may also see this message in the logs as well:

ERROR  [ForkJoinPool-1-worker-3] c.c.S.m.c.JavaMailIMAPClient - Failed to connect to the Store.
javax.mail.MessagingException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

There may be a variety of other SSL related errors as well, such as:

20XX-XX-XX 09:36:30:016 ERROR  [ForkJoinPool-1-worker-11] c.c.S.m.c.JavaMailPOP3Client - [ID:(3121),HN:(mailserver.domain.com)] -> [POP3S|995] Failed to connect to the Store.

Caused by: java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)

Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)

Caused by: java.security.UnrecoverableKeyException: Get Key failed: null

The configuration may have worked in 14.1, but now in 17.1 or higher post upgrade, it no longer works, even using the same certificate files.

Highlighed entries may also present when attempting to import Root CA Certificates from Process Automation or Catalog into SDM

Environment

Service Desk Manager 17.1 and newer

Mailbox in Azure or another similar cloud environment

Cause

Either the NX_KEYSTORE_REF line in the NX.env, or the nx.keystore file is corrupt

When the nx.keystore is created the key should be populated in the NX.env file as the value:
@NX_KEYSTORE_REF=<encrypted line sequence>

Review the NX.env file, if this value is missing then there may have been a problem during the creation of the nx.keystore file.

NX_KEYSTORE_REF is the variable that stores the internal password for the nx.keystore.  This variable is encrypted and not accessible for end users.

For the nx.keystore file, which is located in the NX_ROOT/pdmconf directory, run these two commands (both must be run in sequence)

nxcd bin
pdm_perl pdm_keystore_mgr.pl -list -v

The above will display the contents of the nx.keystore file, using the NX_KEYSTORE_REF variable entry as the password.  SDM Services do not need to be running in order to try the above test.

Resolution

Before making any changes, on each server, take a backup of the following files (copy the files out to a separate directory; do not copy/paste the given file in the same location to make the backup):

- NX.env (in the SDM install directory)

- NX.keystore (in the SDM install's \pdmconf directory)

- client_nx.env (in the SDM install's \site directory)

In addition, collect any Root CA cert files that may need to be reimported, for Catalog, PAM, or Maileater usage.

Stop SDM on all servers

1) Delete the NX.env’s NX_KEYSTORE_REF entry 

2) Delete the file NX_ROOT\pdmconf\NX.keystore 

3) Delete the entry NX_KEYSTORE_REF from NX_ROOT\site\client_nx.env

Note: Repeat this on all SDM servers

4) Restart SDM on the Primary/BG server first, do not start other servers yet

5) Change directory to the SDM install folder's bin directory:

nxcd bin

5) type below via Windows command prompt, changing "<file.cer>" to each certificate used in maileater as well as any other SSL enabled integrations (usually PAM and Catalog):

pdm_perl pdm_keystore_mgr.pl -import <file.cer>

## Repeat the above pdm_perl command to import all needed certificates in the certificate chain for <file.cer>

6) Restart SDM on the given server.

7) Ensure NX.env's NX_KEYSTORE_REF has a valid entry and that it matches the one in NX_ROOT\site\client_NX.env

8) Ensure NX_ROOT\pdmconf\NX.keystore exists 

Optionally run these commands to view the certificates in the NX.keystore:

nxcd bin

pdm_perl pdm_keystore_mgr.pl -list -v

9) Copy the NX_ROOT\pdmconf\NX.keystore to all appropriate SDM servers (example, secondary/app/standby)

10) Restart SDM on all boxes

Additional Information

The NX.keystore is a keystore file that is controlled internally by Service Desk to keep track of certain certificates used to connect and integrate with various products.  It is nominally used to store the root CA cert file used to verify the SSL certificates that the mail host named in maileater uses, as well as any certificates that are deployed to PAM and Catalog. 

If there are other SDM Servers in the same environment (app servers alongside BG Servers), you may also try copying the NX_KEYSTORE_REF entry from a working SDM Server over to the affected server, particularly if the affected server's NX.env lacks the NX_KEYSTORE_REF entry 

The NX.keystore is NOT used to store any SSL certificates that are used to implement SSL on the Tomcat/IIS Server.

How to enable debug logging in for maileater in Service Desk Manager 17.1 and newer:

https://knowledge.broadcom.com/external/article?articleId=98428

Powered by Wolken Software