Product managed Telegraf agent turned to "Unhealthy" state several miniutes after installation

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Product managed Telegraf agent was installed by script following Install/Uninstall an Agent Using a Script on a Linux Platform
• In the beginning the installation state is "Successful" and agent state is "Healthy", "ucp-telegraf" is "running" on Guest OS.
• After several minutes, the agent state is "Unhealthy" on Aria Operations UI, while the "ucp-telegraf" is "dead" on Guest OS when checking with command "systemctl status ucp-telegraf"
• When checking with "journalctl -xe" with Guest OS shell, error "Unit ucp-telegraf.service has failed" can be captured:
  
  MM DD HH:MM:SS <VM hostname> systemd[1]: Failed to start Telegraf.
-- Subject: Unit ucp-telegraf.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- Unit ucp-telegraf.service has failed.

Environment

Aria Operations 8.18.x

Cause

User "arcuser" is not granted with the password-less privilege for "/opt/vmware/ucp/ucp-minion/bin/ucp-minion.sh" 

Resolution

• Follow the instruction in Install/Uninstall an Agent Using a Script on a Linux Platform
• When configure the account privilege, make sure "arcuser" is granted with password-less privilege for "/opt/vmware/ucp/ucp-minion/bin/ucp-minion.sh" as below:
  
  Defaults:arcuser !requiretty
Cmnd_Alias VAPCOMMANDS=/usr/bin/systemctl * ucp-telegraf.service, !/usr/bin/systemctl * * ucp-telegraf.service, /bin/systemctl * ucp-telegraf.service, !/bin/systemctl * * ucp-telegraf.service, /usr/bin/systemctl * ucp-minion.service, !/usr/bin/systemctl * * ucp-minion.service, /bin/systemctl * ucp-minion.service, !/bin/systemctl * * ucp-minion.service, /usr/bin/systemctl * salt-minion.service, !/usr/bin/systemctl * * salt-minion.service, /bin/systemctl * salt-minion.service, !/bin/systemctl * * salt-minion.service, /usr/bin/systemctl * ucp-salt-minion.service, !/usr/bin/systemctl * * ucp-salt-minion.service, /bin/systemctl * ucp-salt-minion.service, !/bin/systemctl * * ucp-salt-minion.service, /usr/bin/netstat, /bin/netstat, /opt/vmware/ucp/ucp-minion/bin/ucp-minion.sh, /usr/bin/systemd-run, /bin/systemd-run
arcuser ALL=(ALL)NOPASSWD: VAPCOMMANDS