Spectrum 23.3.8 broke the UIM integration - Internal Server Error 500

Products

Network Observability Spectrum DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

Since the Spectrum was upgraded from 23.3.4 to 23.3.8 no new UIM alarm in Spectrum OneClick Console. No changes on UIM side.

Enabled the spectrumgtw probe in debug mode:
https://knowledge.broadcom.com/external/article?articleNumber=225136

Restarted the spectrumgtw probe.

The following error is logged in the spectrumgtw.log file:

2024-04-17 11:02:53,654 DEBUG SpectrumGtwScheduler_Worker-1: [Connection] [post] - POST url: http://<OneClick>:80/spectrum/restful/subscription
2024-04-17 11:02:53,702 WARN SpectrumGtwScheduler_Worker-1: [Connection] [post] - Response : InboundJaxrsResponse{context=ClientResponse{method=POST, uri=http://<OneClick>:80/spectrum/restful/subscription, status=500, reason=Internal Server Error}}
2024-04-17 11:02:53,702 ERROR SpectrumGtwScheduler_Worker-1: [Connection] [post] - Exception while processing the POST request
2024-04-17 11:02:53,702 ERROR SpectrumGtwScheduler_Worker-1: [HeartBeatRestImpl] [execute] - SpectrumGtwException in performing heartbeat sync job : com.nimsoft.probe.gateway.spectrumgtw.common.SpectrumGtwException
2024-04-17 11:02:53,703 ERROR SpectrumGtwScheduler_Worker-1: [HeartBeatRestImpl] [execute] - Rest Response:- Status = 500 Error = InboundJaxrsResponse{context=ClientResponse{method=POST, uri=http://<OneClick>:80/spectrum/restful/subscription, status=500, reason=Internal Server Error}}

The following error is logged in the $SPECROOT/tomcat/logs/stdout.log file (RESTful Web Services in debug mode):

Apr 17, 2024 11:02:44.280 (pool-17-thread-3) (RESTFULWEBSERVICES) - (DEBUG) - In GCModelSubscriber ctor for ParsedTypeConstraint
java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: 'boolean com.ca.spectrum.restful.schema.request.GlobalCollection.isGetDeviceNetworkPath()'

Caused by: java.lang.NoSuchMethodError: 'boolean com.ca.spectrum.restful.schema.request.GlobalCollection.isGetDeviceNetworkPath()'

Environment

DX NetOps Spectrum 23.3.8 on Windows 2022 Datacenter

DX UIM 20.4.8 (CU8) - Spectrumgtw 20.40 on RHEL

Cause

This issue is only observed on the Windows platform.

There are new entries in the global-collection section of the $SPECROOT/tomcat/webapps/spectrum/WEB-INF/restful/schemas/Request.xsd file in Spectrum 23.3.8:

<xs:attribute name="getDeviceNetworkPath" type="xs:boolean" default="false" />
<xs:attribute name="getDeviceWebPath" type="xs:boolean" default="false" />

And they were added to the spectrumrest.jar file. But these changes were not added to other jar files by mistake.

The spectrumrest.jar has the latest xsd changes, it must be loaded first.

Resolution

This is the temporarily workaround to load the spectrumrest.jar file first:

Stop the Spectrum Tomcat service
Rename the file $SPECROOT/tomcat/lib/spectrum-alarm-producer-22.2.4.0.jar to xspectrum-alarm-producer-22.2.4.0.jar
Start the Spectrum Tomcat service

The fix to the spectrum-alarm-producer-22.2.4.0.jar file will be available in DX NetOps Spectrum 23.3.10. Once Spectrum OneClick is upgraded to 23.3.10 onwards, remove the old xspectrum-alarm-producer-22.2.4.0.jar file.

Additional Information

Reproduced the issue via Postman.

URL: http://<OneClick_host>:<port>/spectrum/restful/subscription
HTTP Method: POST
Body Context: application/xml
Body:

<rs:subscription-request
xmlns:rs="http://www.ca.com/spectrum/restful/schema/request"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.ca.com/spectrum/restful/schema/request ../../../xsd/Request.xsd "
send-preexisting-instances="false">

<rs:pull-delivery-mode>
<rs:max-notifications>2000</rs:max-notifications> 
<rs:pull-interval>120000</rs:pull-interval> 
</rs:pull-delivery-mode>

<rs:gc-request>
<rs:gc name="UIM_integ" />
<rs:attrs id="0x1134c" notify-of-changes="false"/>
<rs:attrs id="0x12a56" notify-of-changes="false"/>
<rs:attrs id="0x1134b" notify-of-changes="false"/>
<rs:attrs id="0x1102f" notify-of-changes="false"/>
<rs:attrs id="0x11ee8" notify-of-changes="true"/>
<rs:attrs id="0x10e3f" notify-of-changes="false"/>
<rs:attrs id="0x12d7f" notify-of-changes="false"/>
<rs:attrs id="0x129fa" notify-of-changes="false"/>
<rs:attrs id="0x5b60003" notify-of-changes="false"/>
<rs:attrs id="0x10b5b" notify-of-changes="true"/>
<rs:attrs id="0x10024" notify-of-changes="true"/>
<rs:attrs id="0x10023" notify-of-changes="true"/>
<rs:attrs id="0x12981" notify-of-changes="false"/>
<rs:attrs id="0x11f6f" notify-of-changes="false"/>
<rs:attrs id="0x5b60002" notify-of-changes="false"/>
<rs:attrs id="0x22001a" notify-of-changes="false"/>
<rs:attrs id="0x11570" notify-of-changes="false"/>
<rs:attrs id="0x10e40" notify-of-changes="false"/>
<rs:attrs id="0x12dbb" notify-of-changes="false"/>
<rs:attrs id="0x1006e" notify-of-changes="true"/>
<rs:attrs id="0x12c04" notify-of-changes="false"/>
<rs:attrs id="0x23000e" notify-of-changes="true"/>
<rs:attrs id="0x23000d" notify-of-changes="false"/>
<rs:attrs id="0x23000c" notify-of-changes="false"/>
<rs:attrs id="0x12d80" notify-of-changes="false"/>
<rs:attrs id="0x129ac" notify-of-changes="false"/>
<rs:attrs id="0x12d84" notify-of-changes="true"/>
<rs:attrs id="0x12d83" notify-of-changes="false"/>
<rs:attrs id="0x110df" notify-of-changes="false"/>
<rs:attrs id="0x11348" notify-of-changes="false"/>
<rs:attrs id="0x11f7e" notify-of-changes="false"/>
<rs:attrs id="0x12de0" notify-of-changes="false"/>
<rs:attrs id="0x10053" notify-of-changes="false"/>
<rs:attrs id="0x10030" notify-of-changes="false"/>
<rs:attrs id="0x10052" notify-of-changes="false"/>
</rs:gc-request>

</rs:subscription-request>

Where UIM_integ is the Global Collection name used in the UIM integration.