Failure of pruning process causes performance issues on MOM impacting Team Center and WebView

Products

• CA Application Performance Management Agent (APM / Wily / Introscope)
• CA Application Performance Management (APM / Wily / Introscope)
• INTROSCOPE
• DX Application Performance Management

Issue/Introduction

 

APM Team Center and WebView are unavailable because the MOM pruning process failed and this causes the Postgres database size to grow considerably.

 

This is due to the failure of the MOM pruning process caused by the presence of the  appmap_pruning' table which is not cleaned up after previous pruning runs. This causes excessive database growth and therefore causes performance issues with the MOM.

 

The following errors are seen in the IntroscopeEnterpriseManager.log:

[WARN] [pool-23-thread-7] [Manager.ATC.DAO] Creating pruning table failed org.postgresql.util.PSQLException: ERROR: relation "appmap_prunning" already exists
[WARN] [pool-23-thread-7] [Manager.ATC.DAO] Truncating of the pruning table failedorg.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block

Environment

Release : 10.7 Enterprise Manager
Postgres Database only

 

 

Cause

The appmap_pruning table is created as part of the pruning process. Once the pruning is complete, this table is dropped as part of the process. If for some reason, the EM restarts during the pruning process, the table is left in place. When pruning runs in the future, it tries to create this table, but it fails because the table already exists. The above errors can be seen in the EM log on the MOM.

Resolution

As a workaround, the appmap_pruning can be manually dropped. You can use the Postgres tools to execute the following SQL statement:

DROP TABLE appmap_prunning;

Note that the problem is addressed in APM 10.7 HF#52 where the pruning process now checks for the existence of the appmap_pruning  prior to starting. If it exists, it is dropped prior to pruning commencing.