Diego cell vm failing with dynatrace-oneagent unable to start

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

During an upgrade or deployment  diego cell vm is failing state. The dynatrace-oneagent process is unable to start and it is in a failing state

diego_cell/4e52a5c6-a26a-4254-aee3-9329fdd3e794                     -                                        running
~                                                                   blackbox                                 running
~                                                                   bosh-dns                                 running
~                                                                   bosh-dns-adapter                         running
~                                                                   bosh-dns-healthcheck                     running
~                                                                   bosh-dns-resolvconf                      running
~                                                                   dynatrace-oneagent                       failing

Looking into dynatrace-one agent logs located at /var/vcap/sys/log/dynatrace-oneagent/  you noticed that the error below is thrown

ERROR: ld.so: object '/$LIB/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (file too short): ignored. ERROR: ld.so: object '/$LIB/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (file too short): ignored.

Environment

Product Version: 2.11

Resolution

The error is due to dynatrace-oneagent not able to load the needed dependency libraries. The dependency libraries might not exist or it might be corrupted.

To resolve this issue:

1. Delete the failing vm in your IaaS
2. Get the latest manifest : bosh -d <deployment> manifest > manifest.yaml
3. Redeploy using the manifest from step 2 using bosh deploy -d <deployment> <manifest-taken-from step2> This will ensure that the vm will be created and packages including dependency files needed by one agent will be reinstalled.

Note: Dynatrace is a third party add-on and it is one of VMWare's partner service.