ORA-00054 Resource Busy during ZDU Upgrade

Products

• CA Automic Workload Automation - Automation Engine

Issue/Introduction

Introduction

During a Zero Downtime Upgrade (ZDU) to Automic Automation v24.4.4, Work Processes (WP) may fail to start or crash immediately after the initial data load. The system incorrectly attempts a cold start due to a version mismatch, leading to database lock contention.

Symptoms

• WPs fail to establish a connection after starting.
• The WP log populates the following errors:
  
  U00003594 UCUDB Ret: '6' opcode: 'EXEC' SQL Stmnt: 'call UC_Truncate_Table('MQ1WP')'
U00000006 DEADLOCK or Connection to database lost - Rollback handling initiated.
U00003592 UCUDB - Status: '' Native error: '54' Msg: 'ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired'
• The system indicates a version mismatch: Version in Database '24.4.3...', program version '24.4.4...'

Environment

Product: Automic Automation Engine

Version: Upgrading from v24.4.3 to v24.4.4

Database: Oracle 19c

Cause

The issue occurs because the new version of the WP attempts to truncate tables (like MQ1WP) to initiate a start, but it cannot acquire the necessary locks because the resources are still held by the existing system processes or sessions. In Oracle environments with low ddl_lock_timeout settings, the transaction fails immediately with ORA-00054 instead of waiting for the resource to become available.

Resolution

To resolve the contention and allow the ZDU to proceed:

1. Database Adjustment: Have the Database Administrator (DBA) increase the ddl_lock_timeout on the Oracle instance.
   • Recommended Value: 600 seconds (10 minutes).
2. Reset/Restart: Reset the database sessions and restart the Automation Engine processors.
3. Validation: Once the timeout is increased, the WP should be able to wait long enough to acquire the lock.