search cancel

High api response time


Article ID: 253922


Updated On:


CA API Gateway


We are getting a high response time on some API's.

OTK and JWT transaction long transaction time according to Dynatrace, the API gateway spend an amount of time on internal processing, and it impacts on total API response time.


Release : 9.2


BLOCK threads 

"tomcat-exec-executor-732" #167547 daemon prio=5 os_prio=0 tid=0x00007f32b820c000 nid=0xcc7e waiting for monitor entry [0x00007f3355d7c000]

   java.lang.Thread.State: BLOCKED (on object monitor)

at Source)

- waiting to lock <0x00007f338e677da0> (a

at Source)

at Source)

at sun.reflect.GeneratedMethodAccessor645.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(

at java.lang.reflect.Method.invoke(


at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(

at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(

at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(

at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(

at com.sun.proxy.$Proxy103.lookupKeyByKeyAlias(Unknown Source)

at com.l7tech.server.DefaultKeyImpl.lookupKeyByKeyAlias(Unknown Source)

at com.l7tech.external.assertions.jwt.JwtUtils.getKeyFromStore(Unknown Source)

at com.l7tech.external.assertions.jwt.server.ServerDecodeJsonWebTokenAssertion.checkRequest(Unknown Source)



The keystore file can get large.  Reading the keystore file from the database is very expensive.  More time spent on database reading keystore file means those waiting threads are hold for a longer period of time.  This affects performance.

Now we are looking at the method keystore in DatabasePkcs12SsgKeyStore, which has already implemented caching keystorefile mechanism by retrieving keystore file first and then comparing cached keystore file version and database version.  So, when the refreshing keystore period is coming, we can retrieve the keystore file version from the database instead of the actual keystore file, because the version is a very small integer compared to the larger keystore file.

DE353852  Resolution to apply the appropriate patch :

  1. Add caching for private keys and only Create/Decode/Encode Web Token assertions use that. 
    • 9.2 CR9
    • 9.3 CR3
  2. Flip logic for charset encoding resolution, use the faster nio library then fallback to the slower javax.mail library
    • 9.2 CR9
    • 9.3 CR3 
    • 9.4+ upgraded the javax.mail library which fixed this issue.