Spectrum-OI connector is down following rehydration of the OI environment

Products

• DX Operational Intelligence

Issue/Introduction

Spectrum-OI connector is down following rehydration of the OI environment.

Here is the relevant snippet from the log when the alarms stopped flowing. Note that the connector worked initially but then stopped later.

 

FATAL ConnectorSyncScheduler_Worker-9: [client.AbstractHttpClient] [execute] - Error Message: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
ERROR ConnectorSyncScheduler_Worker-9: [manager.DOISync] [pushToJarvis] - Catching
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:198) ~[?:1.8.0_265]
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1967) ~[?:1.8.0_265]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:331) ~[?:1.8.0_265]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:325) ~[?:1.8.0_265]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1688) ~[?:1.8.0_265]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:226) ~[?:1.8.0_265]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1082) ~[?:1.8.0_265]
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:1010) ~[?:1.8.0_265]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1079) ~[?:1.8.0_265]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1388) ~[?:1.8.0_265]

Atempted to restart the connector using the "run.sh start" command and here are the errors from the startup sequence.

ERROR main: [client.ConnectionValidator] [validateTas] - SSL error, import valid certificates.
ERROR main: [client.ConnectionValidator] [validateAll] - TAS connection failed
INFO main: [client.ConnectionValidator] [validateJarvis] - Requested URL : https://doi-nginx.cloud.capitalone.com:443/health/ingestion
ERROR main: [handler.HttpResponseHandler] [handleIOException] - Connection IOException: https://doi-nginx.cloud.capitalone.com:443/health/ingestion
FATAL main: [client.AbstractHttpClient] [execute] - Error Message: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 ERROR main: [client.ConnectionValidator] [validateJarvis] - sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
ERROR main: [client.ConnectionValidator] [validateJarvis] - SSL error, import valid certificates.
ERROR main: [client.ConnectionValidator] [validateAll] - Destination connection failed
INFO main: [client.ConnectionValidator] [validateDOI] - Requested URL : https://adminui.cloud.capitalone.com:443/mdo/v2/aoanalytics/alarms/alarms_spectrum/_search/?size=0&from=0&q=(@doc_type_id:itoa_alarms_spectrum+AND+status:(NOT CLOSED)+AND+product:[email protected]_id:B4F2624C-9B8E-4881-AB1B-B8BD90B2B90C)
ERROR main: [handler.HttpResponseHandler] [handleIOException] - Connection IOException: https://adminui.cloud.capitalone.com:443/mdo/v2/aoanalytics/alarms/alarms_spectrum/_search/?size=0&from=0&q=(@doc_type_id:itoa_alarms_spectrum%20AND%20status:(NOT%20CLOSED)%20AND%20product:Spectrum%20AND%[email protected]_id:B4F2624C-9B8E-4881-AB1B-B8BD90B2B90C)
ERROR pool-6-thread-1: [handler.RetryHttpResponseHandler] [handleIOException] - Handling IOException for /health/ingestion
INFO pool-7-thread-1: [handler.RetryHttpResponseHandler] [handleIOException] - Retrying IOException for /tas/graph/store
ERROR pool-7-thread-1: [handler.RetryHttpResponseHandler] [handleIOException] - Handling IOException for /tas/graph/store
 INFO pool-6-thread-1: [handler.RetryHttpResponseHandler] [handleIOException] - Retrying IOException for /health/ingestion

 

 

Environment

Release : 20.2

Component : CA DOI Foundations

Resolution

Reimport the certificate solved the issue