Active UIM Alarms Constantly Being Cleared and Recreated in Spectrum by the Spectrumgtw Probe
Article ID: 140537
Updated On:
Products
Issue/Introduction
A UIM alarm is getting cleared on its own in Spectrum as per the events. Since the alarm is still alive in UIM the
spectrumgtw probe is re-creating the alarm in Spectrum. After a short period of time, the alarm is again cleared
in Spectrum and re-created. The clearing and re-creation cycle repeats continually.
Cause
It was found that the EventDisp (Event Disposition) files had been modified on the Spectrum side changing
the severity so that it no longer matched UIM's Alarm Severity. The Specrtumgtw probe sees the mismatch
in severity and attempts to rectify this by clearing and re-creating the alarm to try and keep them in sync. The
modification on the Spectrum side prevents the alarm severities from ever matching so this process continues
in an endless cycle while the alarm is still active in UIM
Spectrumgtw log Info
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - InitiateReconcilerAction START
Nov 12 2019 13:35:45,351 [pool-34-thread-1] DEBUG SpectrumGtwAlarmFilter - canBeFiltered start
Nov 12 2019 13:35:45,351 [pool-34-thread-4] TRACE AlarmReconciler - Reconciler current alarm : AlarmAggregratorElem{SpectrumAlarm=Spectrum Alarm [ Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: null Alarm Trouble Ticket Id: null TargetProduct Alarm Id: FR64267104-80190 Alarm Manager : NAS Primary IPV4 Address: null Host Name : uimrobota Source product : NAS LandScape : 11534336 Alarm Source: NAS Alarm Severity: 1 Device Deleted Flag: false targetEntity : null Entity ID : 0xbc02f3 Modified Time : 0 Custom 1 : nullCustom 2 : nullCustom 3 : nullCustom 4 : nullCustom 5 : null UserTag 1 : Windows UserTag 2 : null Status(CauseCount): RootCause , Symptoms : [] ], uimAlarm= Uim Alarm [ Alarm Id: FR64267104-80190 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.20 Alarm Trouble Ticket Id: TargetProduct Alarm Id: null Alarm Manager : NAS Host Name : primaryHubRobot Source product : NAS LandScape : 11534336 Alarm Source: NAS Primary IPV4 Address :10.10.20.25 Alarm Severity: 4 Device Deleted Flag: false Information Warning Alarm deletion Flag: false Severity change for Informational or Warning Alarm: false Alarm severity change flagfalse Entity Id : D1992A49B544EB58D5F4BBBC983D410FE Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : MetricTypeId : 1.2.1:4 Status(CauseCount): RootCause, Symptoms : [] ], aggrEnum=UPDATE}
Nov 12 2019 13:35:45,351 [pool-34-thread-1] DEBUG SpectrumGtwAlarmFilter - canBeFiltered end
Nov 12 2019 13:35:45,351 [pool-34-thread-1] DEBUG SpectrumGtwAlarmFilter - shouldRemoveAlarm start
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - updateAlarms START
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - updateAlarms - update SpectrumAlarm Spectrum Alarm [ Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: null Alarm Trouble Ticket Id: null TargetProduct Alarm Id: FR64267104-80190 Alarm Manager : NAS Primary IPV4 Address: null Host Name : uimrobota Source product : NAS LandScape : 11534336 Alarm Source: NAS Alarm Severity: 1 Device Deleted Flag: false targetEntity : null Entity ID : 0xbc02f3 Modified Time : 0 Custom 1 : nullCustom 2 : nullCustom 3 : nullCustom 4 : nullCustom 5 : null UserTag 1 : Windows UserTag 2 : null Status(CauseCount): RootCause , Symptoms : [] ], uimAlarm= Uim Alarm [ Alarm Id: FR64267104-80190 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.18 Alarm Trouble Ticket Id: TargetProduct Alarm Id: null Alarm Manager : NAS Host Name : primaryHubRobot Source product : NAS LandScape : 11534336 Alarm Source: NAS Primary IPV4 Address :10.10.20.25 Alarm Severity: 4 Device Deleted Flag: false Information Warning Alarm deletion Flag: false Severity change for Informational or Warning Alarm: false Alarm severity change flagfalse Entity Id : D1992A49B544EB58D5F4BBBC983D410FE Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : MetricTypeId : 1.2.1:4 Status(CauseCount): RootCause, Symptoms : [] ], aggrEnum=UPDATE}
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - updateSpectrumAlarm STARTSpectrum Alarm [ Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.18 Alarm Trouble Ticket Id: TargetProduct Alarm Id: FR64267104-80190 Alarm Manager : NAS Primary IPV4 Address: null Host Name : primaryHuba Source product : NAS LandScape : 11534336 Alarm Source: NAS Alarm Severity: 4 Device Deleted Flag: false targetEntity : null Entity ID : D1992A49B544EB58D5F4BBBC983D410FE Modified Time : 1573587264000 Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : Status(CauseCount): RootCause , Symptoms : [] ]
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmServiceFactory - getAlarmProducerService start
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmServiceFactory - getAlarmProducerService returned com.nimsoft.probe.gateway.alarmservice.AlarmProducerProxy
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmProducerProxy - Alarm Uim Alarm [ Alarm Id: FR64267104-80190 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.18 Alarm Trouble Ticket Id: TargetProduct Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Alarm Manager : NAS Host Name : primaryHuba Source product : NAS LandScape : 11534336 Alarm Source: NAS Primary IPV4 Address :null Alarm Severity: 4 Device Deleted Flag: false Information Warning Alarm deletion Flag: false Severity change for Informational or Warning Alarm: false Alarm severity change flagfalse Entity Id : D1992A49B544EB58D5F4BBBC983D410FE Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : MetricTypeId : null Status(CauseCount): RootCause, Symptoms : [] ]updated for proxy com.nimsoft.probe.gateway.alarmservice.AlarmProducerProxy
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - updateSpectrumAlarm END
Nov 12 2019 13:35:45,351 [pool-34-thread-4] TRACE AlarmReconciler - updateAlarms - update targetProductAlarmId in UIM
Nov 12 2019 13:35:45,351 [pool-34-thread-4] TRACE AlarmReconciler - updateAlarms - update alarm in spectrum Uim Alarm [ Alarm Id: FR64267104-80190 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.18 Alarm Trouble Ticket Id: TargetProduct Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Alarm Manager : NAS Host Name : primaryHuba Source product : NAS LandScape : 11534336 Alarm Source: NAS Primary IPV4 Address :10.10.10.18 Alarm Severity: 4 Device Deleted Flag: false Information Warning Alarm deletion Flag: false Severity change for Informational or Warning Alarm: false Alarm severity change flagfalse Entity Id : D1992A49B544EB58D5F4BBBC983D410FE Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : MetricTypeId : 1.2.1:4 Status(CauseCount): RootCause, Symptoms : [] ]
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - updateUIMAlarm START - Uim Alarm [ Alarm Id: FR64267104-80190 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.18 Alarm Trouble Ticket Id: TargetProduct Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Alarm Manager : NAS Host Name : primaryHuba Source product : NAS LandScape : 11534336 Alarm Source: NAS Primary IPV4 Address :10.10.10.18 Alarm Severity: 4 Device Deleted Flag: false Information Warning Alarm deletion Flag: false Severity change for Informational or Warning Alarm: false Alarm severity change flagfalse Entity Id : D1992A49B544EB58D5F4BBBC983D410FE Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : MetricTypeId : 1.2.1:4 Status(CauseCount): RootCause, Symptoms : [] ]
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - updateUIMAlarmExt Start Uim Alarm [ Alarm Id: FR64267104-80190 Troubleshooter: null Alarm Title: Service Control Manager(7038 - None): The wmiApSrv service was unable to log on as DOMAIN\User with the currently configured password due to the following error: The request is not supported. To ensure that the service is configured properly, use the Services snap-in in Microsoft Management Console (MMC). IP Address: 10.10.10.18 Alarm Trouble Ticket Id: TargetProduct Alarm Id: 5dcb0962-96c4-1176-008d-005056bd6b60 Alarm Manager : NAS Host Name : primaryHuba Source product : NAS LandScape : 11534336 Alarm Source: NAS Primary IPV4 Address :10.10.10.18 Alarm Severity: 4 Device Deleted Flag: false Information Warning Alarm deletion Flag: false Severity change for Informational or Warning Alarm: false Alarm severity change flagfalse Entity Id : D1992A49B544EB58D5F4BBBC983D410FE Custom 1 : Custom 2 : Custom 3 : Custom 4 : Custom 5 : UserTag 1 : Windows UserTag 2 : MetricTypeId : 1.2.1:4 Status(CauseCount): RootCause, Symptoms : [] ]
Nov 12 2019 13:35:45,351 [pool-34-thread-4] TRACE y.alarmservice.SeverityLevelMapping - Spectrum level of 4 is 2
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - resolveConflicts START FR64267104-80190
Nov 12 2019 13:35:45,351 [pool-34-thread-4] TRACE y.alarmservice.SeverityLevelMapping - UIM level of 1 is 3
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - Alarm Severity is changed..Performing Reconciliation Action
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG ontroller.AlarmReconciliationAction - resolveSeverityConflicts start
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG ontroller.AlarmReconciliationAction - resolveSeverityConflicts end
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmReconciler - resolveConflicts END
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmServiceFactory - getAlarmProducerService start
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG AlarmServiceFactory - getAlarmProducerService returned com.nimsoft.probe.gateway.alarmservice.SpectrumAlarmProducer
Nov 12 2019 13:35:45,351 [pool-34-thread-4] DEBUG SpectrumAlarmProducer - clear(IAlarm alarm) START
Nov 12 2019 13:35:45,351 [pool-34-thread-4] TRACE SpectrumAlarmPush - Creating alarm batch for deletes. Start index 0. End Index 1
Nov 12 2019 13:35:45,414 [pool-34-thread-4] DEBUG SpectrumAlarmPush - Sending Event Alarm Request
Nov 12 2019 13:35:45,414 [pool-34-thread-4] TRACE Connection - POST(ing): http://OneClickServer:80/spectrum/restful/eventalarm with payload object:
<?xml version="1.0" encoding="UTF-8"?><event-alarm-request xmlns="http://www.ca.com/spectrum/restful/schema/request">
<alarm-delete-list>
<alarm-delete uimid="FR64267104-80190"/>
</alarm-delete-list>
</event-alarm-request>
Environment
Spectrum 10.3.2
UIM 9.0.2
Spectrumgtw Probe 8.67HF2
Resolution
Modify the severity on the UIM Side to the desired level and remove the severity modification
on the Spectrum side so that the alarm matches on both sides.
Feedback
