How to Calculate Number of Nodes in XOG XML Input File To Batch Process

Products

• Clarity PPM SaaS
• Clarity PPM On Premise

Issue/Introduction

Each Clarity environment has an upper bound limit on the maximum number of XML Nodes allowed in any XML Open Gateway (XOG) XML input file used for a XOG read or write request.

This article explains how to calculate the number of nodes in the input file so that it can be broken down into chunks digestible by XOG and batch process requests.

XOG Governor Limit: 
When importing or exporting  many instances using XOG, Out of Memory (OOM) or time-out exceptions can occur.
To prevent these from occurring, configure the Maximum XML Nodes option available under the Application tab in the CSA.

The Maximum XML Nodes specifies the number of XML nodes that can be imported or exported at any given time. The default value is 150,000 for both the On-Demand and On-Premise environments. 

Note: The Maximum XML Nodes is referred to as nodes limit in this document. 
A node is a single unit of Maximum XML Nodes and is the sum of an element and its attributes and does not represent a record.

Environment

SaaS and On Premise environments

Resolution

Nodes are not rows, nodes are every element and attribute value. To see how many nodes exist in the input file, search for the count of  ‘=’
for a rough approximation. Notepad++ or the ‘find’ command used at a command prompt in Windows can be used to quickly obtain such counts. 

The following characters can be counted to estimate the number of nodes.

Count of ‘=’ (attribute assignments): 
Count of ‘/>’ (empty tags): 
Count of ‘</’ (closing tags): 

We strongly recommend that interface design to have the total batch updates stay under the node limitation to produce a scalable and stable interface.
Tests should be performed in a lower environment with the batch method because although the system may allow access the whole set of data,
it will be at the expense of a higher memory footprint, leading eventually to OOM issues.

The files will need to be divided up into a smaller files generated with the assistance of a consultant. The files can then be individually XOG'd in until all the pieces of the batch are complete.