Watch4net: How to use the "parsing-tester.sh" text collector parser test tool; CSV parsing is not working for blank values

Products

• VMware

Issue/Introduction

Symptoms:




This article explains how to use the "parsing-tester.sh" text collector parser test tool, and how to address the issue when CSV parsing is not working for blank values.

Environment

VMware Smart Assurance - Watch4Net/M&R

Resolution

The "parsing-tester.sh" utility is available in /opt/APG/Collecting/Text-Collector/Default/bin for testing the XML parser config when creating text parser xml. The documentation for setting up text parser filter is found in APG-Text-Parsing-Library.pdf, which is installed with the Watch4net software in the /opt/APG/Collecting/Text-Collector/doc directory.

To address the issue of CSV parsing failing for blank values, use the parameter setting skip-empty-tokens="false" as shown in the following example test file:

<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns="http://www.watch4net.com/OutputParsingLibrary" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.watch4net.com/OutputParsingLibrary collectingconfiguration.xsd ">
                <max-threads>1</max-threads>
                <parser-config>
                                <parser-class>com.watch4net.apg.v2.collector.plugins.outputparsing.parsing.generictext.RegexTextParser</parser-class>
                                <tree-type>table-like-tree</tree-type>
                                <incremental-parser>false</incremental-parser>
                                <master-delimiter>\n</master-delimiter>
                                <sub-delimiter skip-empty-tokens="false">
                                                <delimiter>,</delimiter>
                                </sub-delimiter>
                                <parameter name="table-params">
                                                <value>DU-CDR_</value>
                                                <value>^UNIQUE_CALL_ID,CDB_VERSION,CDB_TIMEPOINT,.*</value>
                                </parameter>
                </parser-config>
                <data-formatter-config>com.watch4net.apg.v2.collector.plugins.textoutputcollector.formatting.RawValueFormatter</data-formatter-config>
                <data-handler-config>com.watch4net.apg.v2.collector.plugins.textoutputcollector.handling.DeltaRateRawValueHandler</data-handler-config>
                <data-config>
                                <data-accessor>com.watch4net.apg.v2.collector.plugins.outputparsing.executing.local.FileReader</data-accessor>
                                <stop-on-first-result>false</stop-on-first-result>
                                <host-group-config>
                                                <custom-group>
                                                                <group-name>localhost</group-name>
                                                                <host-config>
                                                                                <host-list>localhost</host-list>
                                                                </host-config>
                                                </custom-group>
                                </host-group-config>
                                <command-configuration>
                                                <command-list>
                                                                <command type="custom">/tmp/test.csv</command>
                                                </command-list>
                                </command-configuration>
                </data-config>
</configuration>

_________________________________________________________

For the following input file'test.csv' :

UNIQUE_CALL_ID,CDB_VERSION,CDB_TIMEPOINT,CALL_REFERENCE_ID,IAM_TIMEPOINT,ACM_TIMEPOINT

1090,1,1337030221,,,

1110,1,1337030371,4fb176d20001abca,,0

1110,1,1337030371,4fb176d20001abcb,,0

1110,1,1337030312,4fb176980001abb3,0,0

1110,1,1337030312,4fb176980001abb4,0,0

1110,1,1337030347,4fb176b60001abbf,0,0

1110,1,1337030347,4fb176b60001abc0,0,0