Metric sink pods fails to start with error "Failed to get current user"

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

Metric sink or log sink resources can be created using the procedure outlined in the TKGI Create Sinks documentation

The metric sink pods fails to start with the following error:

kubectl logs <TELEGRAF-METRIC-SINK-POD>

2021-03-08T16:43:53.610064334Z 2021-03-08T16:43:53Z E! [agent] Service for [inputs.prometheus] failed to start: Failed to get current user - user: Current requires cgo or $USER set in envir
2021-03-08T16:43:53.610074585Z 2021-03-08T16:43:53Z E! [telegraf] Error running agent: Failed to get current user - user: Current requires cgo or $USER set in environment

The metrics sink pod uses the default ServiceAccount token and this is mounted as /var/run/secrets/kubernetes.io/serviceaccount/token on the metrics pod. This token is the user that the pod is referring to in the above log extract. However, the automounting of the credentials for a service account can be disabled by setting automountServiceAccountToken.

Environment

Product Version: 1.9+

Resolution

Review the default ServiceAccount and verify that automountServiceAccountToken is not set to false:

kubectl get sa default -o yaml

If it is set to false, remove "automountServiceAccountToken: false" and restart the metrics sink pod:

kubectl edit sa default
kubectl delete pod <TELEGRAF-METRIC-SINK-POD>