Multi Line Log Analytics events not being processed

book

Article ID: 125445

calendar_today

Updated On:

Products

DX Infrastructure Management NIMSOFT PROBES

Issue/Introduction

When using format rule with log_forwarder to forward multi-line messages, the messages do not appear in log analytics
The following is seen in the Jarvis verifier log on the axa server
 
Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in string value

Cause

The configuration of the Generic log file type in log-parser.conf does not allow the log-parser to handle the new line character added by the multi line configuration

Environment

UIM 8.51
log_forwarder 1.20 \1.30
axa 17.3.0

Resolution

edit the log-parser.conf file found in the DOI setup
/opt/ca/aoPlatform/logparser/logstash-5.5.0/conf
Find the section 
########################################################################### 
# For other log types which is not supported by LA goes to Generic Index # 
###########################################################################
Within this section locate 
# Handling new line character, tabs etc 
mutate { 
gsub => ['message', "[\\]", "/"] 
gsub => ['message', "\"", ""]

and add the line 
gsub => ['message', "\n", " "]
so it looks like
# Handling new line character, tabs etc 
mutate { 
gsub => ['message', "[\\]", "/"] 
gsub => ['message', "\"", ""] 
gsub => ['message', "\n", " "]
Now restart the log-parser
cd to ...../ca/aoPlatform/bin/
then run 
./stopservices.sh lp 
./startservices.sh lp
and verify all is running with 
./healthcheck -la
after this the multi-line messages will be accepted by the log-parser