Enable the JSON parser in vRealize Log Insight 4.6 and later

Products

• VMware Aria Suite

Issue/Introduction

Enable the JSON parser in vRealize Log Insight agents to parse JSON objects (key/value pairs) into corresponding event fields.

Environment

VMware vRealize Log Insight 4.6.x
VMware vRealize Log Insight 4.7.x

Resolution

Windows Agent

1. Stop the Log Insight agent from the command prompt:

net stop LogInsightAgentService

2. In the Windows guest, navigate to Control Panel > System > Advanced system settings > Environmental Variables > System Variables and click New to create a new System Variable with the following configuration:

Variable name: STRATA_AGENT_JSONPARSER_FEATURE
Value: 1

3. Restart the agent from the command prompt:

net start LogInsightAgentService

 

Linux Agent

1. In the Linux guest, stop the Log Insight agent:

service liagentd stop

2. Open /etc/init.d/liagentd in a text editor.
3. Search for the following line:

# Unknown status almost always indicates that the process exists but the pid file is missing

4. Below the else clause, add the following line:

export STRATA_AGENT_JSONPARSER_FEATURE=1 # SHOULD BE ADDED BY THE USER TO BE ABLE TO USE JSON PARSER

Example: 
# Unknown status almost always indicates that the process exists but the pid file is missing
if [ "$status_code" = "$ERRORCODE_IS_RUNNING" ] || [ "$status_code" = "$ERRORCODE_STATUS_UNKNOWN" ]; then
   RETVAL=0
   echo -n "Already started."
else
   export STRATA_AGENT_JSONPARSER_FEATURE=1 # SHOULD BE ADDED BY THE USER TO BE ABLE TO USE JSON PARSER      
   $start_daemon $BinaryPath $StartCommand $*
   RETVAL=$?

5. Save and close the file.
6. Restart the agent:

service liagent restart

 

Agent Group Configuration

1. Log into the vRealize Log Insight UI as local admin.
2. Navigate to Administration > Management > Agents.
3. Expand the groups and click NEW GROUP to create a new group for agents that will use the JSON parser.
4. Click Edit to add the following configuration:

[filelog|json_logs_section]
directory=/tmp/test
include=*.log
parser=json

Note: Edit the directory and include values as needed.

5. Click SAVE NEW GROUP to save the agent group configuration.

Identified JSON objects will be automatically parsed into corresponding event fields key/value pairs and represented as event fields in vRealize Log Insight.

Note: After upgrading LI Agents to 4.6.x and 4.7.x, the JSON parser becomes disabled and will need to be re-enabled.

Additional Information

Example JSON:
{  "firstName": "John",  "lastName": "Smith",  "isAlive": true,  "age": 27,  "address": {  "streetAddress": "380 Interlocken Blvd",  "city": "Broomfield",  "state": "CO",  "postalCode": "80021"  },  "phoneNumbers": [  {  "type": "home",  "number": "1 [phone]"  },  {  "type": "office",  "number": "[phone]"  },  {  "type": "mobile",  "number": "123 456-7890"  }  ],  "children": [],  "spouse": null  }

Example event field results:
firstname
lastname
isalive
age
address_streetaddress address_city address_state address_postalcode
phonenumbers0_type phonenumbers0_number
phonenumbers1_type phonenumbers1_number
phonenumbers2_type phonenumbers2_number
spouse

Example Interactive Analytics:


Impact/Risks:
Extremely large JSON messages could result in server side performance issues.
Tested up to ~500 JSON events without issue, when cluster appropriately sized.