"Timeout occurred" message when generating ACC Diagnostic report
Article ID: 112017
Updated On:
Products
CA Application Performance Management Agent (APM / Wily / Introscope)
INTROSCOPE
Issue/Introduction
The issue manifests itself as the printing of a “timeout” error message in the UI when trying to view certain reports
For example:
NOTE : <x> change based on the report selection
In the log apmccsrv.log:
For example:
“Report with id: <x> can not be found
Timeout occurred “
Timeout occurred “
NOTE : <x> change based on the report selection
In the log apmccsrv.log:
org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Transaction "rolled back" because transaction was set to RollbackOnly.; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Transaction "rolled back" because transaction was set to RollbackOnly. (through reference chain: org.springframework.data.rest.webmvc.json.PersistentEntityJackson2Module$PersistentEntityResourceSerializer$1["content"]->com.ca.apm.acc.configserver.core.domain.DiagReport["agentPackage"])
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:293)
at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:106)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:293)
at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:106)
Environment
ACC 10.7
Cause
The problem is related to defect DE359216
The problem occurs when a request is made to view a report. Config Server loads this report from the database, instantiating a DiagReport instance representing the report which is a managed entity by EclipseLink. The report entry in the database from which the DiagReport instance is created, contains a serialised copy of the Agent entity for which the report was created. It is intended to be a snapshot of the agent at the moment when the report was created. However, as this serialised object was created from an Agent instance, also a managed entity by EclipseLink, when the object is deserialised, EclipseLink realises that it corresponds to a database object and stores it in its L2 cache. Now, whenever the code in the Config Server processing the request requires an instance of this agent, EclipseLink will not go to the database to retrieve it as it already has an Agent instance for this agent in its L2 cache, and so it returns the deserialised version from the DiagReport instance. There are two problems here – first of all, the instance returned may be an old instance – the actual values of the agent may have changed since the serialised version was created when the report was created. Secondly, the code which serialises the Agent, only serialises some of the fields of the Agent. This means that when deserialised, the resultant Agent instance has many null value fields. It is these null values which cause the rollback exception.
Most of the time, when displaying a report, the Config Server does not need to read any values from the Agent instance, and so no error occurs. However, sometimes it does, for example if it detects that there is a newer version of the package than the version from which the report was created, it then needs to work out which version the agent is currently running, and so it needs an instance of Agent. This code does not expect certain fields to be null, and when the deserialised version of Agent from the DiagReport is returned form the L2 cache, a NullPointerException occurs which triggers a rollback exception, which manifests itself in the UI, confusingly, as a timeout error.
The solution was not to serialise the Agent object, but instead create a DTO representing the Agent at the time a report is created that contains only the fields that are needed. This way, when the DTO is deserialised, EclipseLink will not suck it up into its L2 cache thinking it is a managed object.
The problem occurs when a request is made to view a report. Config Server loads this report from the database, instantiating a DiagReport instance representing the report which is a managed entity by EclipseLink. The report entry in the database from which the DiagReport instance is created, contains a serialised copy of the Agent entity for which the report was created. It is intended to be a snapshot of the agent at the moment when the report was created. However, as this serialised object was created from an Agent instance, also a managed entity by EclipseLink, when the object is deserialised, EclipseLink realises that it corresponds to a database object and stores it in its L2 cache. Now, whenever the code in the Config Server processing the request requires an instance of this agent, EclipseLink will not go to the database to retrieve it as it already has an Agent instance for this agent in its L2 cache, and so it returns the deserialised version from the DiagReport instance. There are two problems here – first of all, the instance returned may be an old instance – the actual values of the agent may have changed since the serialised version was created when the report was created. Secondly, the code which serialises the Agent, only serialises some of the fields of the Agent. This means that when deserialised, the resultant Agent instance has many null value fields. It is these null values which cause the rollback exception.
Most of the time, when displaying a report, the Config Server does not need to read any values from the Agent instance, and so no error occurs. However, sometimes it does, for example if it detects that there is a newer version of the package than the version from which the report was created, it then needs to work out which version the agent is currently running, and so it needs an instance of Agent. This code does not expect certain fields to be null, and when the deserialised version of Agent from the DiagReport is returned form the L2 cache, a NullPointerException occurs which triggers a rollback exception, which manifests itself in the UI, confusingly, as a timeout error.
The solution was not to serialise the Agent object, but instead create a DTO representing the Agent at the time a report is created that contains only the fields that are needed. This way, when the DTO is deserialised, EclipseLink will not suck it up into its L2 cache thinking it is a managed object.
Resolution
The fix to this issue is included in 10.7 SP2
Additional Information
10.7 HOTFIXES List:
https://comm.support.ca.com/kb/apm-10-7-hotfixes/KB000105898
https://comm.support.ca.com/kb/apm-10-7-hotfixes/KB000105898
Feedback
Yes
No
Powered by
