DX OI 21.3.1 - Spectrum Data Publisher registration fails with OIUrl FAILED
search cancel

DX OI 21.3.1 - Spectrum Data Publisher registration fails with OIUrl FAILED

book

Article ID: 229473

calendar_today

Updated On:

Products

DX Operational Intelligence

Issue/Introduction

After installed and configured the Spectrum Data Publisher (SDP) it aborts at start with following errors: OIUrl FAILED

Environments details:

a)  DX Platform 21.3.1 using secure routes and certificate has been imported to the SDP cacert as per documentation:

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/21-2/integrating/ca-spectrum-and-analytics/integrate-ca-spectrum-with-ca-digital-operational-intelligence-using-spectrum-data-publisher.html#concept.dita_01d23b21ebdeb3555223a11e764ef8c41f86b5f7_ImportSSLhttpsCertificateintoSpectrumDataPublisher

b) Alarm Reconciliation has been enabled in the conf/ConnectorConfig.xml
..
 <AlarmReconcileConfiguration> <!-- Clears the stale alarms in DOI during service startup. -->
                <Enable>true</Enable>
 </AlarmReconcileConfiguration>
..

 

In SpectrumPublisher log a 500 Internal Server Error message is reported as below :

2021-11-25 17:58:43,191 ERROR main: [handler.HttpResponseHandler] [handleResponse] - HTTP response- Url : https://apmservices-gateway.10.109.32.88.nip.io:443/oipublic/aoanalytics/alarms/alarms_all/_search/?size=0&from=0&q=(status:(NOT%20CLOSED)%20AND%20product:Spectrum%20AND%[email protected]_id:055AF7A9-EA49-42A1-BD0E-74D43D4774BC) , error code: 500 , message: Internal Server Error
2021-11-25 17:58:43,191 ERROR main: [client.AbstractHttpClient] [execute] - HTTP error code: 500
2021-11-25 17:58:43,191 ERROR main: [client.ConnectionValidator] [validateDOI] - OI heartbeat request has failed with status code 500


In the apmservices-gateway log the following SSLException exception is reported:

2021-11-25 18:58:53.428  WARN 1 --- [or-http-epoll-5] r.netty.http.client.HttpClientConnect    : [id:87fd10d3, L:/192.168.196.210:56830 - R:doi-adminui.10.109.32.88.nip.io/10.109.32.88:443] The connection observed an error

javax.net.ssl.SSLException: org.bouncycastle.tls.TlsFatalAlert: certificate_unknown(46)
        at org.bouncycastle.jsse.provider.ProvSSLEngine.unwrap(ProvSSLEngine.java:513)
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
        |_ checkpoint ⇢ com.ca.apm.common.partitioning.PartitioningInterceptor [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.io.IOCounterInterceptor [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.gateway.filters.GatewayAuthenticationFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.io.AccessLogFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.io.ServiceWorkerPoolFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.rest.RequestPathFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.rest.UpgradeFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.io.ServiceCircuitBreakerFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.ca.apm.common.rest.SecureHeadersFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ HTTP GET "/oipublic/aoanalytics/alarms/alarms_all/_search/?size=0&from=0&q=(status:(NOT%20CLOSED)%20AND%20product:Spectrum%20AND%[email protected]_id:055AF7A9-EA49-42A1-BD0E-74D43D4774BC)" [ExceptionHandlingWebHandler]
Stack trace:
                at org.bouncycastle.jsse.provider.ProvSSLEngine.unwrap(ProvSSLEngine.java:513)
                at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679)
                at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:298)
                at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1338)
                at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1234)
                at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1280)
                at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
                at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
                at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
                at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
                at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
                at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
                at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
                at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
                at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
                at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
                at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
                at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
                at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: org.bouncycastle.tls.TlsFatalAlert: certificate_unknown(46)

 

 

Environment

DX Platform 21.3.1

DX Spectrum 21.2.1

Cause

This issue is related to defect DE521769 

Resolution

Solution:

Apply 21.3.1 HOTFIX#1 available from Broadcom Support 

 

Workaround:

1) Login to DX Platform as masteradmin

2) Go to Settings

3) Locate apm.gateway.routes.oipublic property and change uri and version as below:

key: apm.gateway.routes.oipublic

value: { "id": "oipublic", "filters": [ { "name": "RewritePath", "args": { "_genkey_0": "/oipublic/(?<segment>.*)", "_genkey_1": "/oi/v2/oipublic/${segment}" } } ], "predicates": [ { "name": "Path", "args": { "_genkey_0": "/oipublic/**" } } ], "uri": "http://doi-adminui:8080", "order": 0, "metadata": { "version": 2 }}


NOTES
:

a) Increase the version from 1 to 2 while making this change (version should be increased on every change).

b) There won't be any impact due to using the http based service name rather than https route as the communication is internal within the DX cluster. In fact we will change the default settings to use the service name in the next on-prem release.

c) In 20.2.x the /oipublic endpoint was not available and also the corresponding SDP changes were not done to use the reconciliation url. 
 

4) Register the connector using : run.bat/sh reset 

OIUrl check will succeed as below:

Additional Information

https://knowledge.broadcom.com/external/article/190815/dx-aiops-troubleshooting-common-issues-a.html

Attachments