Telemetry Sent to Splunk by OpenTelemetry splunkhecexporter May Lose Sub-Millisecond Timestamp Precision in TAS 6.x and 10.x

Products

• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

Platform operators and application developers using the native OpenTelemetry Collector in Tanzu Application Service (TAS) to forward logs to Splunk may observe that event timestamps lose sub-millisecond precision. In many cases, this appears as millisecond-only precision in Splunk. This can affect sequencing of closely spaced events in high-concurrency environments and reduce timestamp fidelity otherwise available in OpenTelemetry log records

Environment

Tanzu Application Service (TAS) / Elastic Application Runtime (EAR) 6.x and 10.x environments using the OpenTelemetry Collector System Logging feature with splunkhecexporter configured.

Cause

This behavior is tracked upstream in OpenTelemetry Collector Contrib issue #47175. According to the upstream issue, splunkhecexporter reduced timestamp precision before sending data to Splunk HEC by rounding timestamps to millisecond precision in some paths and by converting timestamps through float64 in others, which discarded sub-millisecond precision.

Resolution

A permanent fix will be available in a future TAS/ERT patch release (to be determined) once the underlying otel-collector binary bundled in the platform (tanzu-otel-collector) is bumped to a version containing the upstream fix (v0.149.0 or later).

Check the TAS Release Notes for updates regarding the OpenTelemetry Collector component.

Additional Information

References

• Otel github issue: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/47195