Troubleshooting Database Connection Pool Exhaustion Due to Oracle Tablespace Exhaustion

Products

• CA Strong Authentication

Issue/Introduction

After upgrading to Symantec Advanced Authentication 9.1.5.2 or during migration to new application servers, administrators may encounter application failures where the database connection pool reaches its maximum size. This results in transaction failures and internal server errors (HTTP 1000).

Although the symptoms appear to be a connection pool problem, the root cause is typically Oracle tablespace exhaustion, which prevents successful writing of audit records to the ARWFISSUANCEAUDITLOG table.

Environment

Symantec Advanced Authentication 9.1.5.2
Strong Authentication 9.1.5.2

Cause

Symptoms

• Users receive 1000 (INTERNAL_ERROR) responses.
• Authentication and credential issuance requests fail.
• Application logs show:
  • Arcot Exception, Uncatalogued Error: Could not increase the pool
  • ArDBPool::createNewConnections: Pool [(primary)] is grown by [X] connections. Current total size is [64]
• Informational messages:
  • Current number of threads [116] exceeds the threshold [115]

Restarting Strong Authentication services temporarily resolves the issue, but it reoccurs as the connection pool grows again. Restarting Strong Authentication resets the connection pool and releases held connections. However, if the tablespace issue remains unresolved, new audit writes will again block transactions and cause the pool to fill up.

Oracle database typically reports:

• ORA-01653: unable to extend table ARCOTUSER.ARWFISSUANCEAUDITLOG by 8192 in tablespace ARCOTUSER_DATA
• ORA-30032: the suspended (resumable) statement has timed out
• Older logs may show: ORA-03114: not connected to ORACLE (often after prior database outages or disk space issues)

Resolution

Engage the Database Administrator first to resolve the storage issue before restarting the application.

DBA Actions:

1. Check free space in the ARCOTUSER_DATA tablespace containing ARWFISSUANCEAUDITLOG.
2. Increase tablespace (per organizational policy)
3. Identify and clear any blocked, suspended, or long-running sessions related to the storage failure.

Only after the database issue is fixed:

• Restart Strong Authentication services.

Connection Pool Configuration (Not the Cause)

Typical settings:

• MinConnections=4
• MaxConnections=64
• IncConnections=2
• MaxIdleConnections=64

These values are functioning correctly. Do not increase MaxConnections as a permanent fix — it only masks the problem and allows more blocked sessions to accumulate.

 

Additional Information

Connection pool exhaustion accompanied by Oracle errors ORA-01653 or ORA-30032 is almost always caused by insufficient tablespace preventing writes to the audit log table.

Fix the database storage first. Once resolved, the Strong Authentication connection pool will return to normal operation without any changes to application configuration.