Receiving Connection Reset errors with the JNDI (DYN) connector which does not recover until the JCS is restarted. What do I do?

Products

• CA Identity Manager
• CA Identity Governance
• CA Identity Portal
• CA Risk Analytics
• CA Secure Cloud SaaS - Arcot A-OK (WebFort)
• CLOUDMINDER ADVANCED AUTHENTICATION
• CA Secure Cloud SaaS - Advanced Authentication
• CA Secure Cloud SaaS - Identity Management
• CA Secure Cloud SaaS - Single Sign On

Issue/Introduction

Problem: 

JNDI (DYN) connectors start throwing connection reset errors and do no recover.

Environment:  

All

Cause: 

The JNDI (DYN) connector makes use of Apache Commons GenericObjectPool for connection pooling. By default the JNDI (DYN) connector has minEvictableIdleTimeMillis=600000 and timeBetweenEvictionRunsMillis=6000 which is 10 minutes and 1 minute respectively.

minEvictableIdleTimeMillis specifies the minimum amount of time that an object may sit idle in the pool before it is eligible for eviction due to idle time.

timeBetweenEvictionRunsMillis indicates how long the eviction thread should sleep before “runs” of examining idle objects.

In order for connection recovery to occur there needs to be a period of no activity against the endpoint covering the above time settings else the recovery time will keep starting over. In a real environment there could be continuous attempts to access the endpoint leading to never recovering.

Workaround:

In IM 12.6 releases of the product you can add an additional property of testOnBorrow=true which will enable the validationObject which validates the connection before returning to the client. It simply drops the invalid connection from the pool if it is invalid. Hence the pool can create fresh (valid) connections for the client. So with new property in place you can leave the default values in place for minEvictableIdleTimeMillis and timeBetweenEvictionRunsMillis

1. On all Java Connector Server machines check the C:\Program Files (x86)\CA\Identity Manager\Connector Server\jcs\conf\override\jndi folder to see if a connector.xml file already exists and if not make a copy of the SAMPLE.connector.xml file and rename it to connector.xml instead.

2. On all Java Connector Server machines edit the above mentioned connector.xml file and look for the GenericObjectPoolConfigBeanWrapper bean and add a new property:

<property name=”testOnBorrow”><value>true</value></property>

3. Restart all Java Connector Server service.

 

In IM 12.5 releases of the product the testOnBorrow property will have no effect. In this release of the product the only suggestion would be decrease both the minEvictableIdleTimeMillis and timeBetweenEvictionRunsMillis to much smaller values with the hope that the values are small enough so that the connections may recover during a period of no activity against the endpoint.

1. On all Java Connector Server machines check the C:\Program Files (x86)\CA\Identity Manager\Connector Server\conf\override\jndi folder to see if a connector.xml file already exists and if not make a copy of the SAMPLE.connector.xml file and rename it to connector.xml instead.

2. On all Java Connector Server machines edit the above mentioned connector.xml file and look for the GenericObjectPoolConfigBeanWrapper bean and modify the existing properties:

<property name=”minEvictableIdleTimeMillis”><value>5000</value></property>

<property name=”timeBetweenEvictionRunsMillis”><value>1000</value></property>

3. Restart all Java Connector Server service.

 

Additional Information:

As of the time of this writing the most recent product releases are IM 12.6 SP7 and IM 12.5 SP17.

Engineering is aware of this problem and it is likely that in a future release of the product that the testOnBorrow=true may be set by default without the need to add the property.

Environment

Release:
Component: IDMGR