GemFire 10.1 Prometheus metrics endpoint does not support HTTPS
Article ID: 426473
Updated On:
Products
Issue/Introduction
When using GemFire 10.1 with Prometheus and Grafana, the Prometheus metrics endpoint cannot be secured with HTTPS directly on the GemFire metrics port. Attempts to enforce HTTPS on the metrics endpoint using GemFire HTTP/HTTPS configuration properties do not work.
Environment
- Tanzu GemFire 10.1.x
- Prometheus server
- Grafana dashboards consuming Prometheus metrics
Cause
In GemFire 10.1, the Prometheus endpoint is exposed on a dedicated metrics port configured via the gemfire.prometheus.metrics.port property, and HTTPS is not supported on that metrics endpoint. Starting with GemFire 10.2, Prometheus metrics are exposed via the regular HTTP service port (7070 by default), so the standard GemFire HTTP/HTTPS configuration properties apply and HTTPS can be enabled for the metrics endpoint through the same HTTP service configuration.
Resolution
Upgrade the GemFire cluster to version 10.2 or later, where Prometheus metrics are served on the regular HTTP service port (7070 by default) and can be secured using the standard GemFire HTTP/HTTPS configuration properties. After upgrading, configure HTTPS on the GemFire HTTP service following the 10.2 documentation so that the Prometheus metrics endpoint is served over HTTPS.
Additional Information
Workaround:
If upgrading to GemFire 10.2 or later is not immediately possible, deploy an HTTPS-terminating reverse proxy (for example, Nginx) in front of the GemFire 10.1 Prometheus metrics endpoint. Configure the proxy to accept HTTPS connections from Prometheus/Grafana and forward HTTP traffic to the GemFire metrics port.
References:
- Tanzu GemFire 10.2 Release Notes – Resolved issues and behavioral changes related to metrics exposure and HTTP service port configuration.
- Prometheus Metrics Endpoint - Configuration and behavior of the metrics endpoint in recent GemFire versions.
Feedback
