Missing logmon events

book

Article ID: 143069

calendar_today

Updated On:

Products

NIMSOFT PROBES DX Infrastructure Management

Issue/Introduction

Our users discovered missing tickets for their TWS application events. We checked in both Spectrum and UIM and had verified the last alarms were on 3rd January. We checked logmon configuration and couldn’t find anything wrong. We ran a “test profile” on MaestroEvents profile, and the test returned positive result. We also checked all the logmon, spooler, and controller logs and found no errors. We restarted the robot, created a duplicate profile for MaestroEvents, wrote some false messages, ran a test, and all went well with positive result. Yet there is no single logmon event created.

Cause

- backed up alarm_enrichment queue on the remote hub

Environment

Release : 8.51

Component : UIM NAS

- nas 9.06HF8
- spectrumgw 8.67HF3
- ems 10.21HF1
- logmon 3.70

Resolution

Deployed the latest probe versions for your UIM installation, e.g., in UIM 8.51,

- logmon-4.11-T3
- spectrumgtw-8.6.7-HF2.zip

Then with loglevel set to 5 for nas, logmon, spectrumgtw and ems, with logsize 100000, test again and check the results to see if its resolved. If not, then please attach the log files for the given time period of the test.

Check if the probe, in this case logmon, sends an event and take note of the nimid in the spooler log:

Jan 13 14:56:02:831 [0002] spooler:  data  nimid=RE55133531-03532 nimts=1578945360 tz_offset=18000 source=10.xx.xx.xx md5sum=HEX(16):d37d7a78534de0c9a5a6ad93d56593d7 robot=usxxxxxxxx domain=xxx_UIM_xxx origin=xxx user_tag_1=SunOS 5.10 sparc user_tag_2=Virtual pri=3 subject=alarm prid=logmon dev_id=DE772545932DF0F6B5C970C92A170CA6D met_id=MA

Check IF the alarm is being made invisible, and  also check the local and remote queues (attach/get) if applicable, and make sure no messages are being queued, and also check the nas_alarms, nas_transaction_summary, nas_transaction_log,  using a select statement and you can track the alarm down using the nimid.

example:

select * from nas_alarms where nimid = 'RE55133531-03532'
select * from nas_transaction_summary where nimid = 'RE55133531-03532'
select * from nas_transaction_log where nimid = 'RE55133531-03532'

Also make sure that you can see the alarm in DrNimBUS on that hub the robot belongs to.

The Subject of 'alarm,' means it made it to the alarm_enrichment probe, and then the Subject is changed to alarm2 and passed to the nas.

Check the remote hub queues to make sure that they are not backed up, e.g., alarm_enrichment and nas.

If it ends up being a backed up queue, please note that in the future, you can monitor queues using a custom probe called QueueCheck.

Community reference for queue monitoring via QueueCheck:

https://community.broadcom.com/enterprisesoftware/viewdocument/queuecheck-lua-script-v22?CommunityKey=170eb4e5-a593-4af2-ad1d-f7655e31513b&tab=librarydocuments