Evaluate risk rest api goes down frequently

Products

• CA Risk Authentication
• CA Advanced Authentication
• CA Advanced Authentication - Risk Authentication (RiskMinder / RiskFort)

Issue/Introduction

The Risk REST API endpoint aa-restapi/ca/advancedauth/v1/org/{orgid}/user/{userid}/risk/evaluate goes down intermittently, leading to request timeout errors.

Once the issue occurs, it persists until the WebSphere Application Server is restarted.

During admin credential validation as part of AnA, authentication fails in UDS. When this authentication failure occurs:
1. All subsequent API calls remain stuck at the RESTAPI component on the WebSphere server.
2. Hung threads are continuously observed, leading to degraded performance.

restapi.log
[WebContainer : 26] INFO  risk.impl.ExceptionUserManager(83) [] -> Entering the method : evaluate Risk Rest Service Impl
[WebContainer : 26] INFO  advauth.util.AnaUtil(92) [] -> Get RiskAuth server AnA Configuration from properties file ...
[WebContainer : 26] ERROR risk.impl.ExceptionUserManager(86) [] -> Validate the authToken header value
[WebContainer : 26] ERROR risk.impl.ExceptionUserManager(89) [] -> authToken value is null or empty
[WebContainer : 26] INFO  advauth.auth.AuthServiceImpl(47) [] -> Authenticate admin user to get authtoken after successful authentication
[WebContainer : 27] INFO  advauth.auth.AuthServiceImpl(47) [] -> Authenticate admin user to get authtoken after successful authentication
[WebContainer : 29] INFO  advauth.auth.AuthServiceImpl(47) [] -> Authenticate admin user to get authtoken after successful authentication

arcotuds.log
[WebContainer : 1] : ERROR : anamgmt.ws.ArcotAnARegistrySvcSkeleton : [null] : [null] : [70611] : Authentication failed.
com.arcot.admin.framework.ana.api.AuthenticationException: Authentication failed.
at com.arcot.admin.framework.ana.impl.AuthenticateAndAuthorizeManagerImpl.authenticate(AuthenticateAndAuthorizeManagerImpl.java:201) ~[admin-framework-2.0.jar:?]

SystemOut.log
0000008a ThreadMonitor W   WSVR0605W: Thread "WebContainer : 29" (00000144) has been active for 774631 milliseconds and may be hung.  There is/are 44 thread(s) in total in the server that may be hung.
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:189)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1104)
at com.ca.advauth.util.ServiceConnection.getAnaStubInstance(ServiceConnection.java:373)
at com.ca.advauth.util.AnaUtil.authorizeAnAToken(AnaUtil.java:40)
at com.ca.advauth.risk.services.RiskScoreCalculationRestService.evaluateRisk(RiskScoreCalculationRestService.java:92)

Environment

Symantec Risk Authentication 9.1.5.0028

Cause

As the request is not successful (response not received by the calling application), a few re-tries seem to be triggered for the same. When the call to create authtoken fails, the borrowed connections seem to be not given back to the connection pool, causing the other threads to wait for the connection from the pool.

Resolution

Our recommendation is to apply the patch Symantec-AdvAuth-9.1.5-DE633786-HotFix to resolve this issue. The patch can be downloaded from the KB article as well.