NetOps Portal - Spectrum data source Synchronization Failure after upgrade to 22.2.4

Products

• CA Spectrum
• DX NetOps
• CA Performance Management - Usage and Administration

Issue/Introduction

After upgrading NetOps Performance Management and NetOps Spectrum release from 22.2.3 to 22.2.4, the Spectrum data source in NetOps Portal is failing to synchronize:

 

 

The tomcat log (catalina.out or stdout.log) from Spectrum OneClick server shows these errors - where For input string: "<xxxxxxx>" is the name of a device:

15:19:46.020 [http-nio-8081-exec-13] ERROR org.apache.axis2.transport.http.AxisServlet - processAxisFault() found a null HTTP status from the MessageContext instance, setting HttpServletResponse status to: axis2.http.response.state
Jan 27, 2023 15:27:20.740 (http-nio-8081-exec-15) (NetqosSyncLog) - (ERROR) - pullRequest failed:
Caused by: java.lang.NumberFormatException: For input string: "<xxxxxxx>"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Long.parseLong(Long.java:589)
    at java.lang.Long.parseLong(Long.java:631)
    at com.ca.im.netqos.integration.productsync.pull.PullDeviceAddStageHandler.processBatchedItem(PullDeviceAddStageHandler.java:130)
    at com.ca.im.netqos.integration.productsync.pull.PullDeviceAddStageHandler.processBatchedItem(PullDeviceAddStageHandler.java:30)
    at com.ca.im.netqos.integration.productsync.pull.PullStageHandlerBase.getNextBatch(PullStageHandlerBase.java:238)
    at com.ca.im.netqos.integration.productsync.pull.PullSyncRequest.handleStage(PullSyncRequest.java:103)
    at com.ca.im.netqos.integration.productsync.pull.PullSyncRequest.execute(PullSyncRequest.java:49)
    at com.ca.im.netqos.integration.productsync.SpectrumProductSync2Handler.pullRequestInternal(SpectrumProductSync2Handler.java:583)
    at com.ca.im.netqos.integration.productsync.SpectrumProductSync2Handler.pullRequest(SpectrumProductSync2Handler.java:480)
    at com.netqos.productsync2.webservice.ProductSync2WSDelegate.pullRequest(ProductSync2WSDelegate.java:158)
    at

 

Same message is present in the NetOps Performance Management DMService.log:

ERROR | pool-4-thread-4          | 2023-01-27 07:09:26,298 | com.ca.im.portal.dm.productsync.DataSourcePoller
      |
Data source Spectrum Infrastructure [email protected] encountered an error while processing a sync request.  The problem is in the data source, not CAPC.  Check the logs for the data source to determine the cause of the problem.  The following stack trace shows the context of the sync request:
javax.xml.ws.soap.SOAPFaultException: For input string: "xxxxxxx"
    at org.apache.cxf.jaxws.JaxWsClientProxy.mapException(JaxWsClientProxy.java:195) ~[cxf-rt-frontend-jaxws-3.5.0.jar:3.5.0]
    at

 

Currently there is a device with name “xxxxxxxx” on Spectrum. This is not an SDN device, and there are no SDN VNA Plugins for this landscape containing this device.

However there was a VNA Plugin running for this landscape before, which was removed.

 

Environment

Release : 22.2.4

Cause

The root cause of the issue is that NetOps Spectrum during 'pull' phase is finding a device with a bad SDNID attibute values, which is stopping it from creating a response to NetOps Portal.

This happens because the SDN info was left on the SNMP devices when the VNA Plugin was removed.  

This problem is seen after the upgrade to NetOps 22.2.4 because it is the first release that added the synchronization of SDNID/GWUUID values from NetOps Spectrum data source to NetOps Portal. 

But currently NetOps Spectrum doesn't reset the VNA attributes from the SNMP enabled devices when a device is not managed through VNA anymore. 

Resolution

To identify the issue do the following.

On NetOps Spectrum run a Locater search for Entity UUID attribute 0x13354 for the device and see what is returned. e.g.

./show attributes attr=0x13354 mh=0x1926601
Id          Name                              Iid                  Value
0x13354     Entity_UUID                                    xxxxxxx

Manually remove the string value related to the failing device name (as seen in the logs exception) from the attribute 0x13354 (Entity UUID).

One can notice that all other devices have no value set for this attribute.

After removing the value, it will be like:

./show attributes attr=0x13354 mh=0x1926601
Id          Name                              Iid                  Value
0x13354     Entity_UUID          

Then do a Full sync of the NetOps Spectrum data source on NetOps Portal.

Additional Information

If you run the search and there are no SDN models, try just restarting the OneClick server and clearing the work folder which is essentially the tomcat cache folder:

1.  ./stopTomcat

2.  delete the $SPECROOT/tomcat/work folder

3.  ./startTomcat