Logmon probe won't start

book

Article ID: 187115

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

Logmon probe wont start on a Linux machine. The logmon probe is green and displays a port and a PID but after double-clicking on the probe to open the GUI, in the bottom of the main GUI window, it displays, "The logmon probe is not running."

The log shows 'unable to open data file logmon.dta'

logmon: ****************[ Starting ]**************** 
logmon: * logmon 4.11 
logmon: * Nimsoft Corporation 
logmon: unable to open data file logmon.dta 
logmon: unable to open data file logmon.dta 
logmon: unable to open data file logmon.dta

Cause

- causes may vary, start by referencing all possibilities in this KB article

Environment

Release : UIM v9.0.2
Component : LOGMON v4.11

Robot Environment:
- Linux Redhat v7.6
- logmon v4.11
- Robot: 7.93, 7.97HF4, or 9.20HF5

Resolution

Ensure that all of the documented logmon probe software requirements are met:

The logmon probe requires the following software environment:

- CA Unified Infrastructure Management 8.0 or later
- Robot 7.62 or later (recommended)
  (to enable FIPS encryption) -> Bus (Robot) version 7.80
- Probe Provisioning Manager (PPM) probe version 3.20 or later (required for Admin Console)
- Java JRE 6 or later (required for Admin Console)
- glibc 2.5 or later (required for LINUX platforms)
- Download and install the VS-2017 redistributable package (vs2017_vcredist_x64 and vs2017_vcredist_x86) into the Archive if its not present. (Windows)

- Ensure that the logmon profile name is not very long and that it does not have any periods in the name.

- Ensure that there are no special characters in any logmon profile names

- Make sure that the logmon.dta file exists and that you can write to it. The file MUST be writable. Upon initial logmon probe startup, the logmon.dta is created. When logmon is operating, the file is written to freqently.

WriteDtaFile - getting csDta lock 
logmon: WriteDtaFile - got csDta lock 
logmon: WriteDtaFile - list is empty... 
logmon: WriteDtaFile - releasing csDta lock


- Make sure anti-virus software/scanners/Windows Defender etc. are not interfering with/blocking the operation of the logmon probe. Check the AV logs, and also check the Windows Event logs (Application/System) for any crashes/errors.

- Set the logmon loglevel to 5 and logsize to 10000. Then Deactivate logmon, then Activate it. If the issue reoccurs and/or is reproducible, open a support case if you haven't already and attach the logs.

If there is no change try the following:

1. Create a backup of the logmon probe directory (which includes the logmon.cfg)
2. Delete the probe and the logmon directory on the file system
3. Redeploy the logmon probe (from scratch)

Check the log. If there are no more errors,

- Deactivate the probe.
- Replace the deployed logmon.cfg (basic) with your logmon.cfg.
- Activate logmon and recheck the log.

Here is an example of the contents of a logmon.dta file:

<profiles>
   <Out of Memory Test>
      last_run = 1584071400
      last_filename = C:\temp\Steve\MyTestFile.txt
   </Out of Memory Test>
   <BF9511DE>
      last_run = 1572544684
      last_filename = C:\temp\Steve\MyTestFile.txt
   </BF9511DE>
   <VirtualFailedLoginLog>
      last_run = 1584991373
      last_filename = scripts/RepeatedLoginFailures.sh "3" "1" "REPEATED LOGIN FAILURES"
   </VirtualFailedLoginLog>
</profiles>

On Windows, if you rt-click on the logmon.dta file and choose Properties->Security Tab, it should show that both SYSTEM and Administrators have full access to the file. IF the robot is running as a different user, e.g., a service account, you can try adding full security permissions for that specific user.

Check to make sure your logmon probe obtains a PORT and a PID.

In testing, I was able to reproduce the log entry-> "logmon: unable to open data file logmon.dta"

On a fresh new install/initial deployment of logmon v4.11 I DO see the log entry as soon as the probe is first started:

Mar 25 13:14:36:882 [140309204715328] logmon: ****************[ Starting ]**************** 
Mar 25 13:14:36:882 [140309204715328] logmon: * logmon 4.11 
Mar 25 13:14:36:882 [140309204715328] logmon: * Nimsoft Corporation 
Mar 25 13:14:40:891 [140309204715328] logmon: unable to open data file logmon.dta

But after I configure a logmon profile and watcher, then rt-click to 'Test the profile' the log entries no longer occur even after a restart. I used 'cat' mode to test the profile, 'Match on every run' for example:  after I reconfigure an existing 'out of the box' profile, then test it, the log entries (unable to open data file logmon.dta ), no longer occur.

Mar 25 13:45:32:085 [140154836305728] logmon: ****************[ Starting ]**************** 
Mar 25 13:45:32:086 [140154836305728] logmon: * logmon 4.11 
Mar 25 13:45:32:086 [140154836305728] logmon: * Nimsoft Corporation 
Mar 25 13:45:37:438 [140154836305728] logmon: unable to open data file logmon.dta 
Mar 25 13:47:50:427 [140154836305728] logmon: ****************[ Re-starting ]**************** 
Mar 25 13:47:54:485 [140154836305728] logmon: unable to open data file logmon.dta 
Mar 25 13:47:55:495 [140153813300992] logmon: unable to open data file logmon.dta 
Mar 25 13:48:24:304 [140154836305728] logmon: cmd_runprofile: Executing test profile on profile[BF9511DE] 
Mar 25 13:48:39:585 [140154836305728] logmon: cmd_runprofile: Executing test profile on profile[BF9511DE] 
Mar 25 13:48:49:474 [140154836305728] logmon: ****************[ Re-starting ]****************

On Linux, in the logmon directory you should see these/similar files:

drwx------. 4 root root      221 Mar 25 14:08 .
drwx------. 4 root root       31 Mar 25 12:37 ..
drwx------. 2 root root       34 Mar 25 13:45 factoryTemplates
-rwx------. 1 root root 22323280 Mar 25 13:45 libicudata.so.51
-rwx------. 1 root root  2216272 Mar 25 13:45 libicui18n.so.51
-rwx------. 1 root root  1598240 Mar 25 13:45 libicuuc.so.51
drwx------. 2 root root       25 Mar 25 13:45 locales
-rwx------. 1 root root  3121403 Mar 25 13:45 logmon
-rw-------. 1 root root    11408 Mar 25 13:57 logmon.cfg
-rw-------. 1 root root     6214 Mar 25 13:45 logmon.cfx
-rw-------. 1 root root      223 Mar 25 14:08 logmon.dta
-rw-------. 1 root root     2501 Mar 25 13:57 logmon.log
-rw-------. 1 root root       19 Mar 25 13:57 TempProf.log

So note that you should see the logmon.dta file present and it should have read and write (rw) flags enabled.

Once a successful logmon 'Test profile' run has been performed, the error 'unable to open data file logmon.dta' no longer occurs/cannot be reproduced.

If all else fails, deactivate the logmon probe, then run the probe manually to see if it generates an errors, for example on Linux:

     ./logmon -d 4 -l stdout

Additional Information

Supported Operating Systems for the logmon probe

UIM probe support matrix
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/unified-infrastructure-management/9-0-2/installing/product-compatibility/ca-uim-probes-support-matrix.html

Firewalls
Please also try stopping the Linux firewall and see if the issue is alleviated.
 
Please try the following:
   service iptables stop
 
Current running iptables rules can be viewed with the command iptables -L or use iptables -L -n to also see the port number.
 
   iptables -L
   /bin/systemctl stop iptables.service
 
a) You can FLUSH the rules manually during testing using iptables -F, but the rules are flushed when the iptables is 100% stopped by running the command using systemctl NOT 'service iptables stop.'
 
   /bin/systemctl stop iptables.service
   /bin/systemctl status iptables.service (displays the iptables status)
 
Otherwise if you run service iptables stop it will stop the service but the rules may stay in effect as evidenced by seeing the redirection to /bin/systemctl, e.g., ....Redirecting to /bin/systemctl stop iptables.service despite disabling it with "service iptables stop"
 
Note that the behavior of the iptables stop command may depend on the system/security configuration on the Linux machine.
 
b) You could add a firewall rule on the local Linux machine and keep the iptables enabled. For example:
 
ACCEPT     tcp  --  anywhere             anywhere             state NEW tcp     dpt:48000-48100
 
Ease of configuration depends upon how many Linux machines you need to adjust. It may not be the same config for all so that would need to be checked.
If the firewall/iptables service is truly DISABLED on the robot and you cannot get logmon probe to work, SELinux enforcing may be enabled, so then it may be necessary to disable SELinux on the robot.
 
How to disable SELinux:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux
 
firewalld
 
> service firewalld stop
> systemctl disable firewalld

Logmon - required library files

If logmon is working on one Linux system version e.g., v7.6 and not another, v7.7, check to make sure the required library files are present.

Run the command:

   ldd logmon

for instance on the working versus non-working machine, e.g., RHEL 7.6 machine versus the 7.7 machine.

All of these library files listed below should be present on each machine.

[[email protected]]# cat redhat-release
Red Hat Enterprise Linux Server release 7.6 (Maipo)

[[email protected]]# cd /opt/nimsoft/probes/system/logmon

[[email protected] logmon]# ldd logmon
        linux-vdso.so.1 =>  (0x00007ffce39de000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb2e3cdd000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fb2e3ad5000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fb2e38d1000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fb2e35cf000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb2e33b9000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fb2e30b2000)
        libicuuc.so.51 => ./libicuuc.so.51 (0x00007fb2e2d29000)
        libicui18n.so.51 => ./libicui18n.so.51 (0x00007fb2e290b000)
        libicudata.so.51 => ./libicudata.so.51 (0x00007fb2e11c1000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fb2e0df4000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb2e3ef9000)