TDM Portal: ORA-12516 and HTTP 404 errors

Products

CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

I have successfully installed the CA TDM Portal and I am using Oracle as my repository. The Portal service is started and I am using https://localhost:8443/TestDataManager as my URL in the latest version of my Chrome and Firefox browsers.

The Portal was working fine, but suddenly I began receiving the following error message:
HTTP 404 - /TestDataManager - The resource is not available.

We've enabled debugging using these instructions: TDM Portal: How to enable debugging for TDM Portal

In the resulting log files, we see this error message when trying to start the service:

INFO | jvm 1 | 2017/09/15 16:36:44 | SEVERE: Unable to create initial connections of pool. 
INFO | jvm 1 | 2017/09/15 16:36:44 | java.sql.SQLException: Listener refused the connection with the following error: 
INFO | jvm 1 | 2017/09/15 16:36:44 | ORA-12516, TNS:listener could not find available handler with matching protocol stack

Environment

CA Test Data Manager (TDM)- Web Portal

Cause

One of the most common reasons for this error is the configured maximum number of PROCESSES and/or SESSIONS limitation being reached. When this occurs, the service handlers for the TNS listener become "Blocked" and no new connections can be made. Once the TNS Listener receives an update from the PMON process associated with the Database instance telling the TNS Listener the thresholds are below the configured limit, and the database is now accepting connections connectivity resumes. When the listener believes the current number of connections has reached maximum load, it may set the state of the service handler for an instance to "blocked" and begin refusing incoming client connections with the ORA-12516 error.

More information on the cause of this error can be found here- https://dba.stackexchange.com/questions/110819/oracle-intermittently-throws-ora-12516-tnslistener-could-not-find-available-h

Resolution

This behavior is expected for an Oracle database. If you need to increase the connections, you can change the processes limit. Using connection pooling is probably the best way of fixing this issue, which also reduces the connection acquisition time and levels-up traffic spikes.

When using Oracle with TDM Portal, you can use the below steps to resolve this issue. We highly recommend working with your company's DBA to do this.

The solution is to get rid of all current Portal connections by stopping the TDM Web service, restarting the Oracle database, increasing the process/connection values, restarting the TDM Web Service, re-launching TDM Portal, and verifying logs that error was gone.
Remove all currently used connections.
- Stop the TDM Web Portal Service.
- Verify the number of connections in Oracle manager.
Restart the Oracle database:
- Use the command- shutdown immediate;
- Start the database again.
Increase process values:
- Use the below commands:
  - show parameter processes
  - alter system set processes=300 scope=spfile;
  - alter system set sessions=300 scope=spfile;
    (If you get an insufficient privileges error, request a DBA to help)
Verify the database is still up and running.
- Restart the TDM Web Portal Service.
- Relaunch the TDM Portal in a web browser.
Verify the log files are free of error messages:
- Go to C:\ProgramData\CA\CA Test Data Manager Portal\logs\
- Open startup.log.
- Watch and verify that the microservice start.
Scroll through logs and check for error messages.

Additional Information

If you continue to experience this issue, we recommend working with your company's DBA. Here are some further resources you can use to resolve this issue: