Email alerts and SNMP traps are not sent for repeat trigger conditions

Products

• VMware vCenter Server

Issue/Introduction

To provide a way for users to reduce the throttling of alarms by reducing the alarms reporting frequency.

Symptoms:

• Alarms are configured for email alerts or SNMP traps with state change triggers.
• Initial email alerts and SNMP traps are sent.
• Alarms are not triggered multiple times within a short time period.

Environment

VMware vCenter Server 8.0.x
VMware vCenter Server 7.0.x
VMware vCenter Server 6.7.x
VMware vCenter Server 6.5.x

Cause

The default time period for an alarm is 5 minutes and cannot be changed via the Web Client.

This means, if an alert is triggered multiple times within a 5 minute window, it will only carry out its actions once. This also results in reset rule actions being ignored if the alarm is reset within 5 minutes from when it was first triggered.

Resolution

This 5 minute frequency is expected behavior and serves as a way to prevent spamming of alarm events which could lead to an increased database size. This can result in other complications.

Workaround:
Adjust the frequency of the alarms to 0. Setting the frequency to 0 effectively disables any throttling on alarms and their actions.  

Note: Editing and saving the alarm after applying either of the workarounds below will revert the reporting frequency to its default of 5 minutes again.

To work around this issue, follow one of the options below:

Using PowerCLI

          1. Download the zipped file reset-alarm-reporting-frequency.ps1.gz, that is attached to this KB. This has been provided by VMware Engineering. 

          2. Open the script in a text editor of your choice and replace the text SPECIFY with the relevant info and save it.

For example:
$vcIP = 'VCENTER_IP'
$user = 'VCENTER_SSO_ADMIN_USERNAME'
$pass = 'VCENTER_SSO_ADMIN_PASSWORD'
$alarmName = "ALARM_NAME"

 

3. Run the PowerCLI script.

 

Using the Database

         1. Connect to the vCenter Server database.

For more information, see Connecting to the embedded vPostgres Database in a Windows installed vCenter Server 6.x or Interacting with the vCenter Server Appliance 6.5/6.7/7.0/8.0 embedded vPostgres Database.

 

 

         2. Run the script below to get the XML required to update the alarm:

select name,setting_data from vpx_alarm where name like 'MyAlarm';

You will see output similar to:

 

VCDB=# select name,setting_data from vpx_alarm where name like 'MyAlarm';
 name |                                                                                                                        setting_data

------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------
 MyAlarm| <obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:vim25" versionId="8.0.3.0" xsi:type="AlarmSetting"><toleranceRange>0</toleranceRange><reportingFrequency>300</reportingFrequency></obj>

 

Note: The XML contains unique versionId values for each version of vCenter Server.

 

 

         3. Copy the XML content into the command below, change the reportingFrequency value to 0 and run the command:

this command would update the reportingFrequency of MyAlarm to 0:

UPDATE vpx_alarm SET setting_data='<obj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:vim25" versionId="8.0.3.0" xsi:type="AlarmSetting"><toleranceRange>0</toleranceRange><reportingFrequency>0</reportingFrequency></obj>' WHERE name='My Alarm';
 

         4. Restart the vCenter Server service using the command:

 service-control --stop vmware-vpxd;service-control --start vmware-vpxd;

 

 

 

NOTE: If you modify the alarm definiton, reportingFrequency parameter is reset to default (300 seconds).