Trace View shows incorrect cross-process calls order (downstream agent before upstream agent) compared to Sequence View
search cancel

Trace View shows incorrect cross-process calls order (downstream agent before upstream agent) compared to Sequence View

book

Article ID: 8071

calendar_today

Updated On:

Products

CA Application Performance Management Agent (APM / Wily / Introscope) INTROSCOPE

Issue/Introduction

Using cross-process/cross-jvm transaction tracing to correlate transactions across multiple JVMs/agents. The Trace View shows downstream agent method calls first when it should show upstream method calls first like the Sequence View does.

Environment

APM 9.x, 10.x

Cause

The Trace View will display traces in the order of the agent server timestamp.

If click on each method in the Trace View it will show the Timestamp for the method and it should then be seen that the downstream trace timestamp is earlier than the upstream trace timestamp. That is caused by the system clock on the downstream agent server being behind that of the upstream agent server. The result is that the downstream method appears before the upstream method in the Trace View and is working as designed. 

The Sequence View uses a different mechanism to establish the caller/callee order and was designed specifically to overcome ordering problems in the Trace View due to unsynchronised agent server system clocks.

Resolution

Either ensure that all agent servers have synchronised system clocks or always use the Sequence View to view reliable call order information.

Additional Information

Using Trace and Analyze Events > How to Trace Cross-Process Transaction > Cross-Process Traces Time Alignment

Usually traces display in an order that is based on the system clock where they originated