Grafana displays duplicate data for the same instance when manually setting the UI route for Grafana

Products

• VMware Tanzu Application Service

Issue/Introduction

Grafana displays duplicate data for the same instance as shown in the screenshot below.

And you set the Manually configure for the Grafana UI route.

Resolution

This is a known issue in current healthwatch versions and will be fixed in v2.3.2. Please schedule to upgrade to this fix version if possible.

We have another two temporary workarounds if you cannot wait for the upgrade.

1. Set Automatically configure in TAS for VMs for Grafana UI route: If you are installing Healthwatch on an Ops Manager foundation with TAS for VMs installed, Healthwatch automatically configures a route for the Grafana UI in TAS for VMs. VMware recommends selecting this option when available. You access the Grafana UI by navigating to https://grafana.sys.DOMAIN in a browser window, where DOMAIN is the system domain you configured in the Domains pane of the TAS for VMs tile. For more information, see the TAS for VMs documentation.

2. Disable the master_system_metrics_agent_direct_scrapes job in prometheus configuration in tsdb VM. To disable master_system_metrics_agent_direct_scrapes job you can follow these steps

   1. Login to p-healthwatch TSDB VMs
   2. Run sudo -i to switch to the root user
   3. cd /var/vcap/jobs/prometheus/config/
   4. Open prometheus.yml file and search for master_system_metrics_agent_direct_scrapes
   5. Comment out master_system_metrics_agent_direct_scrapes job

      1. - job_name: master_system_metrics_agent_direct_scrapes
           metrics_path: /metrics
           scheme: https
           tls_config:
             server_name: "system-metrics"
             ca_file: "/var/vcap/jobs/prometheus/config/certs/director_direct_scrape_ca.pem"
             cert_file: "/var/vcap/jobs/prometheus/config/certs/director_direct_scrape_certificate.pem"
             key_file: "/var/vcap/jobs/prometheus/config/certs/director_direct_scrape_certificate.key"
           dns_sd_configs:
           - names:
               - q-s4.*.*.*.bosh.
             type: A
             port: 53035

   6. monit restart prometheus