Logs not reaching Elastic Search after creating sinks that forwards logs to Elastic Search in Tanzu Kubernetes Integrated Edition

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

After creating sinks that forwards logs to Elastic Search, some logs are not reaching Elastic Search in Tanzu Kubernetes Integrated Edition (TKGI).

For more information on sinks, refer to Creating and Managing Sink Resources.

Environment

Product Version: 1.9

Resolution

Log lost may occur when backpressure happens in Fluent Bit. For more information, refer to Fluent Bit: Official Manual - Backpressure.

Increasing Mem_Buf_limit according to the throughput can solve this issue.

Try doubling the size, and continue increasing until issue is resolved.