Error 'Non-compliant scrape target sending blank Content-Type and no fallback_scrape_protocol specified for target' in Gemfire Managment Console

Products

• VMware Tanzu Gemfire

Issue/Introduction

While configuring Prometheus 3.0.0 or above for Gemfire Management Console might receive below error: 
"Non-compliant scrape target sending blank Content-Type and no fallback_scrape_protocol specified for target"

No Content-Type header means it is not compliant. It should be sending something back like Content-Type: text/plain; version=0.0.4.
Latest version of Prometheus needs fallback_scrape_protocol: PrometheusText0.0.4 to be configured explicitly.

Environment

NA

Cause

No Content-Type header means it is not compliant. It should be sending something back like Content-Type: text/plain; version=0.0.4. 
See: https://prometheus.io/docs/instrumenting/exposition_formats/#text-based-format

Until the issue is fixed we can use the setting fallback_scrape_protocol for the target as a workaround. 
The docs are here: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config

Resolution

You'd need to add the setting fallback_scrape_protocol: PrometheusText0.0.4 for that particular target to resolve the issue.

Eg:

cat prometheus.yml 
global:
  scrape_interval: 2s

scrape_configs:
  - job_name: "prometheus"
    fallback_scrape_protocol: PrometheusText0.0.4
    scrape_interval: 2s
    scrape_timeout: 2s
    http_sd_configs:
    - url: "http://<hostname>:<port number>/v1/prometheus-targets"
      refresh_interval: 5s

Additional Information

NA