Agents won't come online in new 21.0 system or after certificate change

Products

• CA Automic Workload Automation - Automation Engine
• CA Automic One Automation

Issue/Introduction

In a new installation of 21.0 or after an update of the keystore/certificate, the agents will not start.  The servername is AUTOMIC01 or fully qualified domain name is AUTOMIC01.fqdn.com.

Java agent logs show the following:
...
20230207/111445.464 -           connection=AUTOMIC01:8443
...
20230207/111446.106 - U02000379 Initiating connection to server 'AUTOMIC01:8443' using WebSocket URI: 'wss://AUTOMIC01:8443/agent'.
...
20230207/111449.454 - U02000385 Web socket error: 'Connection refused: no further information'.
20230207/111449.454 - java.net.ConnectException: Connection refused: no further information
20230207/111449.454 - U02000074 Connecting to system 'AUTOMIC' is not possible.
20230207/111449.470 - com.uc4.ex.cp.InitialConnectionException: Initial connection with endpoint not possible. Please check your configuration.

Windows agent:
...
20230207/111448.534 -           connection=AUTOMIC01:8443
...
[JCPLIST]
20230207/111448.534 -           JCP1=https://AUTOMIC01:8443
20230207/111448.534 -           JCP2=https://AUTOMIC01.fqdn.com:8443
...
20230207/111448.722 - U02000379 Initiating connection to server 'AUTOMIC01.fqdn.com:8443' using WebSocket URI: 'wss://10.10.10.10:8443/agent'.
20230207/111449.814 - U02000313 Communication error with partner 'https://AUTOMIC01.fqdn.com:8443/agent', error: 'No connection could be made because the target machine actively refused it'.

JCP log shows:
...
20230207/111458.530 - 27               ;hostName=
...
20230207/111501.033 - 1                AUTOMIC#CP001     JCP * AUTOMIC01                2317 2023-02-07 17:14:58 2023-02-07 17:15:00 
...
20230207/111540.816 - 32               Jetty: x509=X509@71b4ca62(jetty,h=[automic01],a=[],w=[]) for Server@361dadb4[provider=null,keyStore=file:///E:/####/Automic/Certificate/automickeyfile,trustStore=null]
20230207/111540.816 - 32               Jetty: x509=X509@612d86cf(automic01,h=[automic01],a=[],w=[]) for Server@361dadb4[provider=null,keyStore=file:///E:/####/Automic/Certificate/automickeyfile,trustStore=null]

Environment

Release : 21.0

Resolution

Ultimately, the issue was caused by a mismatch in "case" for the certificate.  The JCP shows the case for the CN:
Jetty: x509=X509@71b4ca62(jetty,h=[automic01],a=[],w=[]) for Server@361dadb4[provider=null,keyStore=file:///E:/####/Automic/Certificate/automickeyfile,trustStore=null]

which has lowercase letters.  The agents are connecting to:
https://AUTOMIC01:8443
or
https://AUTOMIC01.cacu.com:8443

and the server is identifying itself as AUTOMIC01, shown here:
AUTOMIC#CP001     JCP * AUTOMIC01                2317 2023-02-07 17:14:58 2023-02-07 17:15:00

These needed to match up case-wise.  

A good way to do this is to always use either lowercase or uppercase.  If the server identifies itself as lowercase but you'd like it to be uppercase, the certificate can always be uppercase and then use hostname= in the ucsrv.ini file to be the uppercase name for the server.