Retaining ACTIVE Event State for Trap-Based Sources on Domain Manager Restart

Products

VMware Smart Assurance Network Observability

Issue/Introduction

After OI domain manager restarts, all persisted ICS_Notification events transition from ACTIVE to WAS_ACTIVE. For notifications generated from SNMP Trap sources; traps are not re-sent on reconnect, so the state is never restored to ACTIVE automatically.

Environment

Smarts: All Supported OI Suite

Version: 25.x and later

Cause

On restart, the RESTORE_NOTIFICATIONS rule in icoi-init.asl unconditionally calls serverDetached() on every notification, which drives the WAS_ACTIVE transition. Trap-based sources do not re-synchronize, so the state is never corrected after reconnect.

Resolution

NOTE: Following workaround would work on new traps generated after implementing custom asl but would not work on existing trap notification.

A pre-modified icoi-init.asl is provided as an attachment to this KB article. It introduces a configurable variable KeepActiveSourcesOnRestart that, when set to a comma-separated list of event source DomainName values, exempts those sources from the WAS_ACTIVE transition on restart.

Default value: "" (empty) — preserves all existing behavior. Safe to deploy without setting the variable.
No code changes are required, Only the variable value needs to be configured.

Prerequisites

Access to $SM_SITEMOD/conf/icoi/(INCHARGE-OI) on the server.
Know the DomainName of the trap-based source. To find it:

dmctl -s <OI> get ICS_Domain::ICS-Trap-Normalization::DomainName
Example:
dmctl -s INCHARGE-OI get ICS_Domain::ICS-Trap-Normalization::DomainName

For a standard Trap Adapter deployment, the value is typically "Trap Processor".

Implementation Steps

Create the target directory (if it does not exist)
On the OI server, create the following directory: [By default this directory is available]
```
mkdir -p $SM_HOME/local/rules/icoi
```
The typical full path is <SAM_BASE>/smarts/local/rules/icoi/. Files placed here take precedence over the default installation, so the original product file is not modified.
Download and place the modified ASL file
Download the attached icoi-init.asl from this KB article and copy it to the directory created in $SM_HOME/local/rules/icoi/icoi-init.asl
Configure the variable
Open the copied file and locate the following line near the top:
```
default KeepActiveSourcesOnRestart = "";
```
Set it to the DomainName of the trap-based source identified in the Prerequisites step.
Example:
```
default KeepActiveSourcesOnRestart = "Trap Processor";
```
Note: For multiple sources, use a comma-separated list with no spaces:
Example:
```
default KeepActiveSourcesOnRestart = "Trap Processor,MyOtherTrapSource";
```
This is the only line in the file that needs to be edited. No other modifications are required.

Restart the domain manager

sm_service stop <OI_SERVICE>
sm_service start <OI_SERVICE>

Post restart, All the notifications from the specified source under variable "KeepActiveSourcesOnRestart" will have the EventState as "ACTIVE", verify it as:

dmctl -s <OI> get ICS_Notification::<NotificationName>::EventState
Example:
dmctl -s INCHARGE-OI get ICS_Notification::<NotificationName>::EventState

Notes

The KeepActiveSourcesOnRestart value is case-sensitive and must exactly match the DomainName attribute of the source domain.
Multiple sources are comma-separated with no surrounding spaces: "Source1,Source2".
The default empty string preserves all standard WAS_ACTIVE behavior — this change is fully opt-in and backward compatible.
This file is placed in local/rules/icoi/ which takes priority over the standard installation path via SM_SITEMOD resolution. The original product file under $SM_HOME/rules/icoi/ is never modified.

Additional Information

Rollback

Remove the placed file from $SM_HOME/local/rules/icoi/ and restart the domain manager. The server will revert to the original product file with no changes:

rm $SM_HOME/local/rules/icoi/icoi-init.asl
sm_service stop <OI_SERVICE>
sm_service start <OI_SERVICE>

Attachments

icoi-init.asl get_app