ASM Log Output to CSV Script

Products

• CA Application Performance Management (APM / Wily / Introscope)
• CA Application Performance Management Agent (APM / Wily / Introscope)
• CA Application Performance Management SaaS

Issue/Introduction

The ASM v3 API commands replaced rule_log with log. This no longer had a direct way to export log output as CSV. (Was doing JSON only.) This script provides a way to do this with the log command.

 

Note: This script provided AS-IS . That means that there is no support on this or updates planned. You are free to use and update the script.

Resolution

The script requires 2 parameters - source and destination files:

./ApiGetRuleLog.py -s log.json -o log.csv
where log.json is the source file with the json result from APIv3 GET /log call and log.csv is the final file that will be exported. There are also two more options -d (delimiter) and -q (quote) that can be used to fine tune the CSV format:

./ApiGetRuleLog.py -h
usage: ApiGetRuleLog.py [-h] -s SOURCE -o OUTPUT [-d {comma,semicolon,tab}] [-q {all,string,minimal,none}]

Translate APIv3 GET /log output to rule_log CSV format

options:
  -h, --help            show this help message and exit
  -s SOURCE             source file
  -o OUTPUT             output file
  -d {comma,semicolon,tab}
                        CSV delimiter, default is comma
  -q {all,string,minimal,none}
                        use quotes always or only when needed, default is none (backward compatibility)

Additional Information

From Engr:

Why we went with JSON:
The json format is better for this purpose as it allows us to structure the data. E.g. the assets can be empty (as in the example below) or can contains several files.

[
  {
    "id": "<id>",
    "monitor": {
      "id": "<id>",
      "name": "<name>"
    },
    "location": {
      "id": "180",
      "name": "Tokyo"
    },
    "folder": {
      "id": "51271",
      "name": "My monitors"
    },
    "start": "<start-time>",
    "duration": 3,
    "interval": 60,
    "monitor_type": "https",
    "result": {
      "code": 0,
      "description": "Matched",
      "type": "ok",
      "performance": "good"
    },
    "metrics": {
      "rtime": 1,
      "ctime": 244,
      "ptime": 743,
      "dtime": 743,
      "dsize": 424
    },
    "assets": null,
    "browserMessages": null
  }
]

It is not possible to do this in the csv file.

Engr found a few problems in the format above, e.g. the monitoring station ID is not there (only location ID), or monitor_type should be nested in the monitor info block. 
They will create a defect for that and we fix it in this or (more likely) next release.

 

However, I've prepared a python script that loads the json file and exports it as the csv file as rule_log does. 
There are a few columns missing but I do believe it is not a problem. These columns were omitted compared to the original rule_log output:

deleted: current status of the monitor is "deleted" (monitor property, not needed here with every record)
repeat: it was always 1 (deprecated)
utime: user metrics, different meaning for different monitor types, it was always very confusing (deprecated)
type: numeric type that describes if the monitor is single/multi, sync/async, it is an internal value, not useful for customers
alerts: number of alerts sent (performance penalty)
loc: short code (e.g. a1) of the station that performed the check (deprecated - replaced with group which is id of the location)
ipaddrs: IP address(es) of the station(s) that performed the check
id: calculated id based on timestamp and monitor ID - used for pagination (deprecated)
Also the long format with monitor output (parameter full=y in rule_log) was not included. If they need that it is not difficult to extend it.