Enable and Tune Archiving and Purging feature

Products

• CA Process Automation Base
• Process Automation Manager

Issue/Introduction

After deploying Process Automation, the Archive and Purge feature needs to be enabled for overall health of the Orchestrator.    The single largest contributor to performance issues within Process Automation is directly related to not Archiving and Purging completed instances.    As the number of rows grow in both the Runtime database, and the Runtime Archive database tables, overall performance decreases, to the point where the engine can no longer complete instance activity.

It is vital ensure the runtime and archive database tables are kept to as small as possible.   To this effect we recommend being as aggressive as possible with the archive and purge settings.

Environment

All Process Automation 4.x releases

Resolution

Instructions for enabling Archive and Purge for completed Processes:

Log into Process Automation as Pam Admin.

1. Navigate to the Configuration Browser.
2. Expand the Orchestrators Pallet from the lower left corner.
3. Highlight the Orchestrator Name, right click and Lock.
4. Click on the Policies Tab
5. Specify values for Minimum Days of Process History, No of Days to Keep Archived Data, and ensure Option to Purge Archive Data is set to Purge Archived Data Daily.
6. Save the modifications.
7. Click the Delete Archived Instance button - specify the date range, and purge the Archive to clear all processes that where Archived prior to the upgrade as they will not have any Purge schedule configured.

Discussion of Archive and Purge configuration

To ensure the overall runtime database is kept to as small a size as possible we recommend being as aggressive as possible with the archive and purge settings.
For example if you determine that you need 7 days worth of historical process data you could set Minimum Number of Days of Process History to 1, and Option to Purge Archived Data to Purge Daily, and Number of Days to Keep Archived Data to 6;
Or you could set Days of Process History to 3 and Option to Purge to 4; or you could set Minimum Number of Days of Process History to 7, and set Option to Purge Archived Data to Purge Data without archiving, which would simply Purge the process completely after 7 days.   

There are a variety of additional configuration options that can be used to ensure that there is a minimum of instance history kept for review, while ensuring that the majority of instances are quickly archived and purged from the database.

1. Minimum Number of Days of Process History

Default is 2 days.   This is the setting that tells PAM how long to wait after an instance has completed before Archiving.   In 4.2 Sp02 and later, instances are rolled up and moved to the archive table immediately upon completion.  This setting now controls how long that completed instance remains in the Current view, and indicates to PAM how long it should wait before moving the instancing into the purging engine.

2. Minimum Number of Failed Instances

3. Minimum Number of Finished (Completed, Aborted) Instances

Default for both is 0.  These settings allow you to override the other time based Archive policies forcing a certain number of failed, Completed, or Aborted, instances to remain in the Current view - this would ensure an admin can review if necessary.    

4. Option to Purge Archived Data

What happens to Archived data - default is Do Not Purge Archived Data meaning that even once archived the instance will remain until manually purged.   We recommend the 'Purge Archived Data Daily' option.
    
5. Number of Days to Keep Archived Data

Default is 10.  Indicates how long after an instance has been marked as archived, and displayed under the Archive area of the Operations tab before it is purged entirely from the database. 

Additional Information

Please note, these settings are applied at run-time.  The configuration at the time an instance is initiated is stored within that Instance run and will not be updated if the configuration changes later.

Changes to the policies will only be applied to new instance runs.

If your Process Automation engine has been running for without the Archive and Purge configured, you will need to use the Right Click / Archive option from the Operations view to push instances into the Archive:

And then on the Configuration> Orchestrator > Policies Tab, use the Delete Archived Instances button to delete the existing Archived Instances:

It is recommended that you start with 1-3 month blocks to purge as this will impact the database and can impact overall Orchestrator performance.