Grafana dashboard show gaps in VM health dashboard in large scale cluster
Article ID: 297929
Updated On:
Products
VMware Tanzu Application Service for VMs
Environment
Product Version: 2.0
Resolution
Checklist:
The VM health dashboard relies on on metric data from system-metric-agent on each VM, but only one system-metric-scraper on clock_global instance has to scape all system-metric-agent at specified interval. The scrape timeout is 1 second, which could be triggered very often when the scraper reach out hundreds of VMs concurrently, thus as the result, some metric scraping fail and those failed periods will be display on Health Watch grafana dashboard as gaps.
The issue has been fixed in system-metrics-scraper-release v4.0.2, and will be included in TAS 2.11.46.
On Health Watch grafana dashboard, healthy instance groups are displayed as green bar, while unhealthy instances_groups(when any instance in the group is unhealthy) as red bar.
In very large scale of clusters, there will be gaps instead of green or red bar.
The VM health dashboard relies on on metric data from system-metric-agent on each VM, but only one system-metric-scraper on clock_global instance has to scape all system-metric-agent at specified interval. The scrape timeout is 1 second, which could be triggered very often when the scraper reach out hundreds of VMs concurrently, thus as the result, some metric scraping fail and those failed periods will be display on Health Watch grafana dashboard as gaps.
The issue has been fixed in system-metrics-scraper-release v4.0.2, and will be included in TAS 2.11.46.
Feedback
Yes
No
Powered by
