search cancel

Logmon crash (segfault) from a particular profile on Linux servers

book

Article ID: 230000

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM) CA Unified Infrastructure Management SaaS (Nimsoft / UIM) Unified Infrastructure Management for Mainframe

Issue/Introduction

We have several logmon probes crashing on our Linux servers. The common point is that they all have the same profile .

We see crashes (segfaults) in the messages logs. Over 483 events since the 18/08.

Sample errors from logmon.log:

logmon: continue_scan - command /apps/nimsoft/custo/scripts/illumio_status_connectivity.sh is still running
...
logmon: lgm: NULL from ptScanLineExNonBlock !
...
logmon: lgm: read the line: [Oct  5 11:58:20 s00vl9956454 nrpe[4996]: Error: Request packet type/version was invalid!]
...
…nrpe[30716]: Error: Request packet type/version was invalid!
…nrpe[30716]: Client request was invalid, bailing out...
…<hostname redacted>kernel: logmon[19303]: segfault at 18 ip 000000000044d6a8 sp 00007f6cb4ae8b00 error 4 in logmon[400000+26c000]

…nrpe[30947]: Error: Request packet type/version was invalid!
…nrpe[30947]: Client request was invalid, bailing out...
…nrpe[30965]: Error: Request packet type/version was invalid!

Environment

Release : 20.3

Component : UIM LOGMON

OS: RHEL 6 Update 10

Cause

- logmon command configuration

- environment variables cannot be set on the logmon command line, they must be included/set in a batch file or script

Resolution

Ran the command without setting the environment variables in the logmon command window.

For example:

   /opt/illumio_ven/bin/AgentSend connectivity_test -j;

The command ran successfully as per the logmon.log and the /var/log/messages file no longer showed that the segfault was thrown.

If environment variables need to be set to accommodate differences in machine environments, along with running the command that would have to be included in a batch file/script, the batch file / exe / or script can then be called through logmon.

As per the logmon techdocs:

(Run command on match: allows you to run commands or batch files with optional parameters, when the log file text matches the expression.)

"Optional parameters" include commands and arguments but presetting environment variables in the logmon command window is not supported by the probe.

Customer included everything they needed inside the script including the grep and used the exit code monitoring in logmon and this avoided the segfault.