Emails from alarms not being delivered.

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

No alarms seen in the alarm view since March 11, 2017.

Environment

- emailgtw v2.83- nas v8.42- alarm_enrichment v8.42

Cause

Unknown but seemed to be due to alarm_enrichment probe being in some form of a hung state.

Checking history we saw two alarms on 3/13 and 3/14.

nas and AE queues showed up in the hub status window as green and processing alarms but I noticed a delay with the AE probe in terms of processing, e.g., delay then all alarms sent in one chunk.

Resolution

I noticed that the alarm_enrichment probe had low memory, out-of-the-box settings at 64m and 1024m respectively, so I increased the memory to 2048/4096. The AE probe works best when the memory is optimized in that manner. Then we cold started the probe. All alarms started flowing and emails were sent out but the nas probe seemed unstable, taking a long time to sync.

Also all alarms had time stamps for today at the time of our fixing the issue instead of when the alarms truly originated which was previous to today.

I reorganized the nas database (housekeeping), via the Status window in the nas probe via rt-click->Reorganize, and that seemed to help temporarily.

Since there were a few fixes to the AE probe in nas 8.43 we deployed the hotfix but the nas was still taking very long to sync the NiS Bridge.

I had to redeploy the nas probe and then it responded very quickly without delays.