Vertex out-of-sync Alarm Issue

Products

• DX Operational Intelligence

Issue/Introduction

A service indicates that it isn't healthy (i.e., it's not 100% and green).  However, there are no active alarms associated with this service.

Resolution

Once the environment variable is updated, you can either check immediately or wait for 2 hours sometimes when the tenant cache is not initialized before the state remediator jobs start, as the remediator jobs run every 2 hours by default.
Follow these steps to validate the changes:

1. After the environment variable is updated, the pod will restart.
2. Navigate to the pod's log directory: `soa/logs/servicestatemanager-<>/`.
3. Run the `ls -lart` command to check the file size. Ensure that the stateremediator file size is not zero.
4. Open the state manager log file and search for the following log entry:
   `"Remediation successfully completed for tenant id"`

   This log entry will indicate how many vertices were corrected by the remediator job, specifically where the state was not synchronized with the alarm state.