Healthwatch deployment does not complete because the Grafana VM is failing when deployed with TKGi version 1.11
search cancel

Healthwatch deployment does not complete because the Grafana VM is failing when deployed with TKGi version 1.11

book

Article ID: 298686

calendar_today

Updated On:

Products

VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

When deploying Healthwatch 2.1.1 on OpsMan with TKGi version 1.11, the deployment of Healthwatch fails because the Grafana service cannot start.

The error in Grafana logs is shown below:
cp: cannot copy a directory, '/var/vcap/jobs/grafana/config/provisioning/dashboards/pks/', into itself, '/var/vcap/jobs/grafana/config/provisioning/dashboards/pks/selected'

Further checks shows that in the /var/vcap/jobs/grafana/config/provisioning/dashboards/pks/' folder there are versions 1.8, 1.9, and 1.10 but it's missing 1.11 which is causing the copy dashboards script to fail.

Environment

Product Version: 1.11

Resolution

This problem has been addressed by the dev team and is expected to be fixed in the next Healthwatch release.


Workaround 1 (the most convenient way):

If you have TKGi 1.11, from the Healthwatch tile under Grafana Configuration, change Discover Product Dashboards from dynamic to manual and specify for the TKGi version to be 1.10.

This way the system will not perform automatic discovery but will explicitly define the dashboard that has to be used. 

Another workaround for this problem during the deployment is described below.

There is a certain amount of time where the system will retry to start the service. If this time is passed, the deployment will fail. It is a five minute window. Follow the deployment progress and when you see that the system is starting Grafana, follow the steps below:
 
  • ssh to the Grafana VM
  • Run the following commands:
    • sudo -i 
    • cp -R /var/vcap/jobs/grafana/config/provisioning/dashboards/pks/1.10 /var/vcap/jobs/grafana/config/provisioning/dashboards/pks/1.11
    • chown -R root:vcap /var/vcap/jobs/grafana/config/provisioning/dashboards/pks/1.11
    • chown -R root:vcap /var/vcap/jobs/grafana/config/provisioning/dashboards/pks/selected
  • watch monit summary 

Confirm that the Grafana service is running and that the deployment has progressed.