MOI raw metric data collection in JSON format from Kafka OI-TIMESERIES-RAW-PAYLOAD-RECORDS topic

Products

• Mainframe Operational Intelligence

Issue/Introduction

Attached is a script (extractTSD21_1653417037087.sh) for raw TSD JSON data collection from the Kafka queue via the OI-TIMESERIES-RAW-PAYLOAD-RECORDS topic.  This is the Time Series Data(TSD) in JSON format that is being received by MOI for ingestion so it is the actual metric data values. 

This script is designed to collect only five minutes of metric data for reloading and analysis by MOI Engineering.

Environment

Release:   2.1

Resolution

Instructions to capture the metric JSON data:

     *  Download the extractTSD21_1653417037087.sh script to the MOI appliance into the /tmp directory.

     *  Execute the following Linux commands on the MOI Appliance:

                chmod +x extractTSD21_1653417037087.sh

                ./extractTSD21_1653417037087.sh

The script will collect TSD JSON data for 5 minutes and then terminate on its own. Once the script completes the data collection,  it creates a 'tsd.json' file in the /tmp directory. 

Please upload the resulting 'tsd.json' file to MOI Support via a Support Case.   Please also upload a screenshot of the collection script output for verification by MOI Engineering.

Here is a sample of the JSON data being collected:

▒ZDM.CA.SYSVIEW.ZOS.16.00990.GSVX.ZOSCPU.fe033079-aa82-4a82-9fd2-82836d9689b5SYSVIEW16.00990▒{"type":"TSD","start":"2022-05-24T12:04:00-04:00","end":"2022-05-24T12:04:30-04:00","entity":[{"class":"ZosSystem","name":"CA31","entity":[{"class":"ZosCpuCore","name":"ALL","metrics":[ {"name":"CPU%LPAR","value":"69"}]},{"class":"ZosCpuCore","name":"CP","metrics":[ {"name":"CPU%LPAR","value":"85"},{"name":"CPU%RSHR","value":"117"},{"name":"CPUPWT  ","value":"500"}]},{"class":"ZosCpuCore","name":"SP","metrics":[ {"name":"CPU%LPAR","value":"36"}]},{"class":"ZosCpuCore","name":"IFA","metrics":[ {"name":"CPU%LPAR","value":"0"},{"name":"CPU%RSHR","value":"0"},{"name":"CPUPWT  ","value":"0"}]},{"class":"ZosCpuCore","name":"IIP","metrics":[ {"name":"CPU%LPAR","value":"36"},{"name":"CPU%RSHR","value":"154"},{"name":"CPUPWT  ","value":"150"}]},{"class":"ZosCpu","name":"ALL","metrics":[ {"name":"CPU%    ","value":"59"}]},{"class":"ZosCpu","name":"CP","metrics":[ {"name":"CPU%    ","value":"96"}]},{"class":"ZosCpu","name":"SP","metrics":[ {"name":"CPU%    ","value":"23"}]},{"class":"ZosCpu","name":"IFA","metrics":[ {"name":"CPU%    ","value":"0"}]},{"class":"ZosCpu","name":"IIP","metrics":[ {"name":"CPU%    ","value":"23"}]}]}]CA31

Example run of the metric data collection script:

[root@hostname tmp]# ./extractTSD21_1653417037087.sh

Collecting TSD data ...

PID: 28405

Tue May 24 11:03:59 CDT 2022

Collecting TSD data for 5 minutes ...

Press CTRL+C anytime to exit.

Tue May 24 11:08:59 CDT 2022

./extractTSD21.sh: line 23: 28405 Terminated              docker exec -e JMX_PORT="" kafka_kafka_1 ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic OI-TIMESERIES-RAW-PAYLOAD-RECORDS > ${fname}

Finished.

TSD Data saved as tsd.json

Captured 647 lines in tsd.json.

 

[root@hostname tmp]# ls -ltr

total 8196

-rwxr-xr-x. 1 root root     658 May 24 11:03 extractTSD21.sh

-rw-r--r--. 1 root root 5155750 May 24 11:08 tsd.json