HTTPS configured Data Aggregator fails to start with after upgrade to 25.4.4 or later

Products

Network Observability CA Performance Management

Issue/Introduction

After upgrading the PM Data Aggregator from to 25.4.7, the dadaemon service starts but fails approximately 2 minutes later. The REST API does not come live.

In the /opt/IMDataAggregator/data/logs/netops-data-aggregator.log there is an error similar to:

Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
YYYY-MM-DDTHH:MM:SS,### | ERROR | main | SpringApplication | framework.boot.SpringApplication 857 | - - | | Application run failed
org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:408)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:394)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:586)
at java.base/java.lang.Iterable.forEach(Unknown Source)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:364)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:310)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:1010)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:631)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:752)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:439)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:318)
at com.ca.im.dm.aggregator.DataAggregator.main(DataAggregator.java:29)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:31)
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat server
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start(TomcatWebServer.java:251)
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:44)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:405)
... 21 more
Caused by: java.lang.IllegalArgumentException: standardService.connector.startFailed
at org.apache.catalina.core.StandardService.addConnector(StandardService.java:220)
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.addPreviouslyRemovedConnectors(TomcatWebServer.java:310)
at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start(TomcatWebServer.java:236)
... 23 more
Caused by: org.apache.catalina.LifecycleException: Protocol handler start failed
at org.apache.catalina.connector.Connector.startInternal(Connector.java:1115)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
at org.apache.catalina.core.StandardService.addConnector(StandardService.java:217)
... 25 more
Caused by: java.lang.IllegalArgumentException: Error creating SSLContext
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:115)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:78)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:224)
at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1431)
at org.apache.tomcat.util.net.AbstractEndpoint.start(AbstractEndpoint.java:1514)
at org.apache.coyote.AbstractProtocol.start(AbstractProtocol.java:644)
at org.apache.catalina.connector.Connector.startInternal(Connector.java:1112)
... 27 more
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at java.base/java.security.cert.PKIXParameters.setTrustAnchors(Unknown Source)
at java.base/java.security.cert.PKIXParameters.<init>(Unknown Source)
at java.base/java.security.cert.PKIXBuilderParameters.<init>(Unknown Source)
at org.apache.tomcat.util.net.SSLUtilBase.getParameters(SSLUtilBase.java:532)
at org.apache.tomcat.util.net.SSLUtilBase.getTrustManagers(SSLUtilBase.java:461)
at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:265)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:113)
... 33 more

Environment

Upgrade to 25.4.4 or later with HTTPS confiugred Data Aggregator

Cause

The dadaemon service fails to fully start because the root and intermediate certificates are missing from the Data Aggregator truststore (/opt/IMDataAggregator/config/ssl/truststore), though they may be present in the keystore

Resolution

Add the root and intermedaite certificates to the truststore and restart the dadaemon services:

systemctl stop dadaemon
systemctl start dadaemon