Offline processes don't watch correctly

book

Article ID: 211349

calendar_today

Updated On:

Products

CA Release Automation - Release Operations Center (Nolio)

Issue/Introduction

We need to revise the problem with the Dashboard --> Procces Report --> Offline Procceses because we can't watch them.

Sometimes this work correctly but the most part of time the proccesses don't load in this dashboard. We noticed when we restart the NAC service we see some new records getting move to offline processes.

Cause

The Purging service of RA is single threaded, so only one instance of the purging process may be executed simultaneously. This service does the following actions:

    • Copies finished execution jobs from online tables to the offline tables.
    • Removes online execution jobs that were previously copied to the offline tables from the online tables.
 
So, no new execution jobs will be copied to the offline table, unless all the already copied jobs will be removed from the online tables which means no new jobs will appear in the offline processes report.
Issues I found in the customer's logs/DB:
 
The cause of problem is poor performance of the operation with removing records from the exec_flows table and some error can cause death of purging service. In this particular case we noticed error in nolio_perf_mon_nac.log.* as below causing purge service to get terminated and not resume until restart of NAC service.
 
Oracle throws the following error:
ORA-30036: unable to extend segment by 8 in undo tablespace 'UNDOTBS1'

Environment

Release : 6.6

DB Type: Oracle 

Component : CA RELEASE AUTOMATION CORE

Resolution

Steps to follow

1: Please run the below query against the RA DB and it will improve the performance of the purging service and improves online jobs removal operation. The below script will create a DB index.

# FOR ORACLE

DECLARE L_INDEXES_COUNT INTEGER;
BEGIN
SELECT COUNT(*)
INTO L_INDEXES_COUNT
FROM USER_INDEXES
WHERE UPPER(INDEX_NAME) = UPPER('IX_EXEC_FLOWS_JOB_ID');

IF (L_INDEXES_COUNT = 0) THEN
EXECUTE IMMEDIATE 'CREATE INDEX IX_EXEC_FLOWS_JOB_ID ON EXEC_FLOWS(JOB_ID)';
END IF;
END;
/

2: The restart of NAC is not required after index is created.

3: This new index will help purging service to remove copied online execution jobs and this will help to stabilize the online/offline processes report.

4: To check the progress of the offline job movements can be tracked running the following query. The amount returned by this query must go down.

select count(e.id) from EXECUTION_JOBS e where e.major = 'Final'

Additional Information

  1. There is no rollback query for this and this rollback query is not needed - the index that is created with the provided script must exist.
  2. The purging service in Nolio was fixed in scope of defect NOLIO-3623. Please check the release notes of upcoming cumulative patches to determine fix patch version of this defect.
  3. In case if removal of Index is required due to any concern, customer DBA can remove the same easily.