How to Calculate the Nodes in Your XOG XML Input File and Batch Process Your Requests

book

Article ID: 104078

calendar_today

Updated On:

Products

CLARITY PPM FOR ITG CLARITY PPM FEDERAL Clarity PPM SaaS - Application Clarity PPM On Premise

Issue/Introduction

Each CA PPM environment has an upper bound limit on the maximum number of XML Nodes allowed in any XML Open Gateway (XOG) XML input file used for a XOG read or write request. This article explains how to calculate the number of nodes in your input file so that you can break it down into chunks digestible by XOG and batch process your requests.

XOG Governor Limit: 

When you import or export many instances using XOG, you can encounter Out of Memory (OOM) or time-out exceptions. To handle these issues, you can configure the Maximum XML Nodes option available under the Application tab in the CSA. The Maximum XML Nodes specifies the number of XML nodes that can be imported or exported at any given time. The default value is 150,000 for both the On-Demand and On-Premise environments. 

Note: The Maximum XML Nodes is referred to as nodes limit in this document. 
A node is a single unit of Maximum XML Nodes and is the sum of an element and its attributes. 

Environment

CA PPM 15.1+ SaaS and On Premise environments

Resolution

Nodes are not rows, nodes are every element and attribute value. To see how many nodes are in your file search it for the count of ‘=’ for a rough approximation and then add some. Notepad++ or the ‘find’ command used at a command prompt in Windows can be used to quickly obtain such counts. 

You can count in your file. 

Count of ‘=’ (attribute assignments): 
Count of ‘/>’ (empty tags): 
Count of ‘</’ (closing tags): 

We strongly recommend that you design your interface to batch updates to stay under the node limitation to produce a scalable and stable interface. You would need to change how you are pulling/pushing the data. 

Although the system may allow you to access the whole set of data, it will be at the expense of a higher memory footprint, leading eventually to OOM issues. If/When your data grows to the point where 150,000 won't work, you'll be in the same boat once again. 
The file can be divided up into a smaller files (a task for the customer or consultant helping them) and then each file can be XOG’d in individually until all the pieces of the batch are complete. The file should be divided into smaller pieces to XOG in successfully.