NetOps OIConnector - Failed ingesting inventory (iteration 1 of 1) to TAS for NetOps Portal tenant id Error: 500: {"error":{"code":301,"message":"DATABASE_ERROR"

Products

• DX OI SaaS
• DX Operational Intelligence

Issue/Introduction

We have deployed NetOps OIConnector, we can see topology and metrics in DX OI but we have noticed the below ERROR messages that related to 2 of our NetOps tenants in the OIConnector log

 [ERROR] 2023-12-20 08:12:24.503 [TASIngestionBatcher-1] InventoryTaskImpl - Failed ingesting inventory (iteration 1 of 1) to TAS for CAPC tenant id : <_default_ OR your tenant name(s)> Error: 500: {"error":{"code":301,"message":"DATABASE_ERROR"

What is the reason of the errors and how can we fix it?

Environment

DX NetOps Event Manager 23.3.3, Data Aggregator 23.3.3.7

DX OI SaaS

Cause

The issue is related to NetOps OIConnector defect DE587214

Analysis from Development:

 

There are duplicate group strings as shown below repeated 100s of times.

 

    "All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custom Collections:OI Collection;All Groups:Custo

 

This is added to all interface attributes causing the size of the vertices to go into multiple MBs.

This is a bug in the connector for populating group names with Custom Collections. 

 

Currently such large vertices are causing memory issue for TAS also.

 

Resolution

RESOLUITON:

Upgrade to NetOps OIConnector 3.12

WORKAROUND: 

Disable the interfaces coming from NetOps Portal.  When interface attribute is removed, the interface level metrics will also be disabled by the connector and so customer would be able to use only device level inventory and metrics for now.

1. Login to the machine where the connector is configured.
2. Browse to the install location and conf folder (/opt/CA/OIConnector/conf)
3. Open config.xml file in the Edit mode
4. search on "includeInterfaceInventory" and change the following property to false this will completely disable sending interface information as inventory:
       <!-- Set to true to send interfaces as inventory -->
        <property name="includeInterfaceInventory" value="false"/>
5. Save it
6. Restart the connector after making this change to have it take effect.
   service caperfcenter_oi restart

 

Additional Information

https://knowledge.broadcom.com/external/article/190815/aiops-troubleshooting-common-issues-and.html