Clarity application connection broken after switching between primary and secondary DB servers

Products

• Clarity PPM On Premise

Issue/Introduction

Clarity PPM is not automatically reconnecting to the database after a planned or unplanned failover event in an Oracle Data Guard Disaster Recovery (DR) setup. When the Database Team performs a switchover (e.g., Primary DB server to Secondary, and Secondary to Primary), the Clarity application's existing database connection fails, rendering the application inoperable.

Customers are seeking assistance on how to configure Clarity's database connection string to include multiple database hosts (Primary and Standby) within the same connection string. The goal is to enable automatic reconnection to the newly active primary database without requiring manal intervention at the application layer, potentially using a format similar to this:

(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=server1)(PORT=1525))(ADDRESS=(PROTOCOL=TCP)(HOST=server2)(PORT=1525))(CONNECT_DATA=(SERVICE_NAME=sample_service_name)))

Additionally, guidance is needed on how to implement this multi-host connection string within Clarity's application properties to mitigate the impact of future database switchovers or failovers.

Environment

Clarity 16.3.3

Cause

Clarity PPM application services, by design, do not support fully automated database failover in a cross-datacenter DR scenario without manual intervention at the application layer. Even when Oracle Data Guard successfully switches database roles and the new primary becomes available, the Clarity PPM application itself requires a restart of its services to recognize and establish a new connection to the active primary database.

While Oracle's JDBC drivers support multi-host connection strings (as shown in the Issue description) designed for client-side failover, Clarity's application architecture holds onto the initial connection or connection pool state, and does not dynamically re-evaluate the connection string or establish new connections to the alternate host automatically after a database role change without a service restart.

Resolution

Here's a simplified breakdown of the process:

1. Stop Clarity PPM Services: You will need to manually stop all Clarity PPM application services.
2. Update Database Connection: Update the database connection string in Clarity's configuration (e.g., in CSA or properties.xml) to point to the new primary database host.
3. Start Clarity PPM Services: Manually start all Clarity PPM application services.

Regarding using a database alias for DR:

Setting up a robust database alias (like a TNS alia or a Data Guard Broker-managed service name) that always resolves to the active primary database is a good practice. This would simplify Step 2 above, as you wouldn't need to manually change the host/IP details in Clarity's configuration (JDBC connection string). However, Steps 1 and 3 (stopping and starting Clarity services) would still be required for the application to pick up the new connection.

Clarity product documentation provides various JDBC URL examples, these primarily focus on high availability within a single data center (like Oracle RAC) or standard connections.

Reference : Connection string examples