ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

Threshold profiles don't seem to be working.

book

Article ID: 214169

calendar_today

Updated On:

Products

CA Performance Management - Usage and Administration DX NetOps

Issue/Introduction

It doesn't look like the threshold profiles are working since we migrated to a new install of Performance Management. Threshold Monitoring is enabled per KB article: https://knowledge.broadcom.com/external/article/37241/threshold-events-stopped-appearing-in-dx.html

<?xml version="1.0"?>
<ThresholdMonitoringConfigurationList>
<ThresholdMonitoringConfiguration version="1.0.0">
<ID>3505</ID>
<ThresholdMonitoringEnabled>true</ThresholdMonitoringEnabled>
<RecoveryIntervalInMinutes>15</RecoveryIntervalInMinutes>
<ThresholdMonitoringLimiterEnabled>true</ThresholdMonitoringLimiterEnabled>
<PercentOfPollCycleThreshold>80</PercentOfPollCycleThreshold>
</ThresholdMonitoringConfiguration>
</ThresholdMonitoringConfigurationList>

Cause

EM database is corrupt, and unable to be repaired via mysqlcheck

Environment

All supported DX NetOps Performance Management releases

Resolution

Use the following steps to drop/delete the EM data base and create a new one.

NOTES:

  1. Remove the Event Manager (EM) Data Source (DS)
    1. Go to Administration->Data Sources->Data Sources
    2. Select the Event Manager Data Source
    3. Delete the Event Manager Data Source
  2. Stop the EM service.
  3. Drop the EM database.
    1. Go to /opt/CA/MySql/bin
    2. Enter the MySql prompt using: ./mysql -uroot -p
      • Enter the password when prompted
    3. Run this command to drop the EM DB: drop database em;
    4. Note: If you are unable to drop the DB due to further errors take these steps:
      1. Stop the MySql service.
      2. Stop the four Performance Center processes
      3. Delete the /opt/CA/MySql/data/em folder
      4. Start the MySql service.
  4. Run the following npcshell.sh command.
    • Replace <MySql_HostName> with the host name or IP of the MySql host.
    • Replace <MySqlPasswd> with the complex MySql password. Enclose the password in single quotes if the password contains special characters.
    • /opt/CA/PerformanceCenter/Tools/bin/npcshell.sh initem --production --dbhost <MySql_HostName> --dbpass <MySqlPasswd>
  5. Start four Performance Center services
  6. Add the EM DS to the Portal.
    1. Go to Administration->Data Sources->Data Sources
    2. Add the Event Manager Data Source

Review Threshold configuration and see if Events begin appearing again when Threshold Rules are violated.

Additional Information

The following errors were seen when attempting to repair and drop the em database.

mysqlcheck -u root -p --auto-repair --all-databases
ERROR | pool-5-thread-1          | 2021-04-22 09:00:51,559 | com.ca.im.portal.em.dao.mysql.query.GenericBatchInsertWithReturn
      | exception while executing batch and retrieving keys (SQL: INSERT IGNORE INTO event_identifiers(ProducerID, LocalEventID, State, OccurredOn) values (?,?,?,?))
java.sql.BatchUpdateException: Can't find file: 'event_identifiers' (errno: 2 - No such file or directory)

Repairing tables
em.event_identifiers
Error    : Can't find file: 'event_identifiers' (errno: 2 - No such file or directory)
status   : Operation failed
em.event_properties
Error    : Got error 122 from storage engine
error    : Corrupt

mysql> drop database em;
ERROR 1010 (HY000): Error dropping database (can't rmdir './em', errno: 39)