REST process only starts with matching keystore and key passwords

Products

• CA Automic Workload Automation - Automation Engine
• CA Automic One Automation

Issue/Introduction

sslenabled=1 for REST process in 21 requires keystorepassword and keypassword to be identical

Steps to reproduce:

1. Setup a keystore that has a key in it - make sure the keystorepassword and keypassword are the same (I did password and password)
2. Setup ucsrv.ini file with the correct info for TLS
3. Start your system as normal to see that the keystore and password works - doublecheck in the AWI that all functions are working as expected
4. Stop the REST process
5. In the [REST] section, set sslenabled=1
6. Start the REST Process
7. Notice that the REST process starts correctly and uses the keystore from the TLS section as shown in line:
   20220324/042135.413 - 37               Jetty: x509=X509@19444ed8(FQDN.domain,h=[FQDN.domain],w=[]) for Server@669110b7[provider=null,keyStore=file:///C:/Tools/certs/keyfile,trustStore=null]
8. Stop the REST process
9. Update the keystore password to something different like password1
10. Update the [TLS] section of the ucsrv.ini file to reflect the new keystorepassword - feel free to stop and start the full system except REST to show that JCP and everything else works fine
11. Start the REST process

Expected results:
REST process should start as it did in step 7 above and search should be fully functional in AWI

Actual result:
Sometimes the REST process ends abnormally with error:
20220324/042947.728 - 35     U00045014 Exception 'java.io.IOException: "keystore password was incorrect"' at 'sun.security.pkcs12.PKCS12KeyStore.engineLoad():2116'.
20220324/042947.728 - 35     U00045015 The previous error was caused by 'java.security.UnrecoverableKeyException: "failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption."' at 'sun.security.pkcs12.PKCS12KeyStore.engineLoad():2116'.
20220324/042947.728 - 35     U00003620 Routine 'com.automic.rest.server.RestServerComponent' forces trace because of error.

Other times, the REST process will start as expected, but error will be thrown when trying to do things like Search which will return:

No search results available because the REST-API server is not running
The AWI Search API could not respond to your request.

Deleting agents from client 0 Administration perspective will show something like:

No search results available because the REST-API server is not running
Can't connect to any REST-endpoint, please check if at least one JCP is running and is reachable via network.

Environment

Release : 21.0.2

Component : AUTOMATION ENGINE

Resolution

This is a bug that was fixed with 21.0.3.

Workaround 1:
Update password for both keystore and key to be the same
Update ucsrv.ini to use the password specified

Workaround 2:
Create new keystore with same cert, use the same password for the keystore and key
Copy ucsrv.ini file to ucsrvrest.ini
Update ucsrvrest.ini to have the new password under [TLS] for keystorepassword and keypassword settings
Keep original pwds and keystore in ucsrv.ini
Update smgr for REST process to use ucsrvrest.ini using the -i start parameter