How to change Healthwatch 2 alert manager notification from HTTPS to HTTP in Operations Manager

Products

• Operations Manager

Issue/Introduction

In an alert manager notification, the first line is the link to the Grafana dashboard which comes with a default HTTPS URL. You may want to adjust this to make it HTTP to avoid having to consistently change the URL for the dashboard from the email alert hyperlink. 

For example:
 

View a Grafana graph of the expression that triggered this alert
^^^ Above URL is set for https://grafana-pas-systemdomain.com/explore? ^^^

View this alert in Alertmanager (Requires a BOSH SSH Tunnel)
Summary: A certificate is expiring
Description: At least one certificate (BOSH Trusted Certificate S/N: XXXXXXX) on your foundation is going to expire within 30 days.
Details:
alertname: ExpiringCertificate
instance: 10.x.x.x:9090
job: healthwatch-pas-exporter
product: BOSH Trusted Certificate S/N
prometheus_deployment: p-healthwatch2-xxxxxxxxxxx
scrape_instance_group: cert-expiration-exporter

This articles explains how to configure the URL type as HTTP.

 

Environment

Healthwatch 

Operations manager

Resolution

To work around this issue, follow the steps below: 

1. First validate your settings by adjusting the root URL from https to http in your Operations Manager (Ops Manager) -> Healthwatch -> Grafana Configuration and Apply Changes.

2. Validate HTTP is being used in the Grafana initialization file:

grafana/033747ea-f3a1-4579-a141-4ef71c0c4fdb:~# grep protocol /var/vcap/jobs/grafana/config/grafana.ini 
protocol = http

3. If these settings are correct, please follow the instruction below:

a. SSH into each Time Series Database (TSDB) VM and go into sudo mode (sudo -i) so you can review this file:

/var/vcap/jobs/prometheus/config/alerting.rules.ym

b. vim the file above and when it is open you should be able to type a colon(:) and paste the following command:

%s/grafanaUrlPrefix: https/grafanaUrlPrefix: http/g

Hit enter for this command to globally switch all of the prefixes to http.

Once this is done, you need to go on all TSDB VMs and restart the prometheus service.

monit restart prometheus

Note: These steps must be followed any time Healthwatch redeploys the TSDB VMs.