Update thresholds for the Ambari alert "Ambari Agent Disk Usage"
search cancel

Update thresholds for the Ambari alert "Ambari Agent Disk Usage"

book

Article ID: 295022

calendar_today

Updated On:

Products

Services Suite

Issue/Introduction

Ambari has a default alert "Ambari Agent Disk Usage" that monitors disk usage across all nodes in the cluster. The alert can be viewed at Ambari Web GUI > Alerts > "Ambari Agent Disk Usage":

However, the alert cannot be modified via the Ambari Web GUI. The API needs to be used to modify the alert thresholds. This article describes the steps that can be taken to modify the threshold values for this alert to activate.


Environment


Resolution

Log on to the Ambari node as the user, root, and take these steps to change the threshold values:

1. Find the cluster name (in this case, the cluster name is "amb171hawq"): curl --user admin:admin http://localhost:8080/api/v1/clusters/":

[root@amb171hawq ~]# curl --user admin:admin http://localhost:8080/api/v1/clusters/
{
 "href" : "http://localhost:8080/api/v1/clusters/",
 "items" : [
 {
 "href" : "http://localhost:8080/api/v1/clusters/amb171hawq",
 "Clusters" : {
 "cluster_name" : "amb171hawq",
 "version" : "PHD-3.0"
 }
 }
 ]
}[root@amb171hawq ~]#

2. List all the configuration settings: curl -u admin:admin -X GET http://<AMBARI_SERVER>:8080/api/v1/clusters/<CLUSTER_NAME>

[root@amb171hawq /]curl -u admin:admin -X GET  http://localhost:8080/api/v1/clusters/amb171hawq
{
  "href" : "http://amb171hawq:8080/api/v1/clusters/amb171hawq",
  "Clusters" : {
    "cluster_id" : 2,
    "cluster_name" : "amb171hawq",
    "health_report" : {
      "Host/stale_config" : 1,
      "Host/maintenance_state" : 0,
      "Host/host_state/HEALTHY" : 1,
      "Host/host_state/UNHEALTHY" : 0,
      "Host/host_state/HEARTBEAT_LOST" : 0,
      "Host/host_state/INIT" : 0,
      "Host/host_status/HEALTHY" : 0,
      "Host/host_status/UNHEALTHY" : 1,
      "Host/host_status/UNKNOWN" : 0,
      "Host/host_status/ALERT" : 0
    },
    "provisioning_state" : "INSTA

<...>

    {
      "href" : "http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/42",
      "AlertDefinition" : {
        "cluster_name" : "amb171hawq",
        "id" : 42,
        "label" : "Ambari Agent Disk Usage",
        "name" : "ambari_agent_disk_usage"
      }
    },
    {
      "href" : "http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/43",
      "AlertDefinition" : {
        "cluster_name" : "amb171hawq",
        "id" : 43,
        "label" : "Ambari Agent Heartbeat",
        "name" : "ambari_server_agent_heartbeat"
      }
    },
    {
      "href" : "http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/44",
      "AlertDefinition" : {
        "cluster_name" : "amb171hawq",
        "id" : 44,
        "label" : "Ambari Server Alerts",
        "name" : "ambari_server_stale_alerts"
      }
    }
  ],
  "artifacts" : [ ]
}
[root@amb171hawq /]#
3. Locate the "Ambari Agent Disk Usage" setting and get the URL for the configuration (under href). In this case, the URL would be: http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/42.


4. Look up the configuration details using the link found above:

[root@amb171hawq /]# curl -u admin:admin -X GET  http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/42
{
  "href" : "http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/42",
  "AlertDefinition" : {
    "cluster_name" : "amb171hawq",
    "component_name" : "AMBARI_AGENT",
    "description" : "This host-level alert is triggered if the amount of disk space used on a host goes above specific thresholds. The default values are 50% for WARNING and 80% for CRITICAL.",
    "enabled" : true,
    "id" : 42,
    "ignore_host" : false,
    "interval" : 1,
    "label" : "Ambari Agent Disk Usage",
    "name" : "ambari_agent_disk_usage",
    "scope" : "HOST",
    "service_name" : "AMBARI",
    "source" : {
      "parameters" : [
        {
          "display_name" : "Minimum Free Space",
          "description" : "The overall amount of free disk space left before an alert is triggered.",
          "name" : "minimum.free.space",
          "value" : "5.0E9",
          "type" : "NUMERIC",
          "units" : "bytes",
          "threshold" : "WARNING"
        },
        {
          "display_name" : "Warning",
          "description" : "The percent of disk space consumed before a warning is triggered.",
          "name" : "percent.used.space.warning.threshold",
          "value" : "0.5",
          "type" : "PERCENT",
          "units" : "%",
          "threshold" : "WARNING"
        },
        {
          "display_name" : "Critical",
          "description" : "The percent of disk space consumed before a critical alert is triggered.",
          "name" : "percent.free.space.critical.threshold",
          "value" : "0.8",
          "type" : "PERCENT",
          "units" : "%",
          "threshold" : "CRITICAL"
        }
      ],
      "path" : "alert_disk_space.py",
      "type" : "SCRIPT"
    }
  }
}
[root@amb171hawq /]#
5. Copy the output from the step above.

6. Remove the "href" line from the output copied in the step above.

7. Modify the values that need to be changed in the "Value" and the "Description" fields.

8. Submit the changes to Ambari by putting the JSON created above into a curl command. The JSON should be inserted after the "PUT -d ' " and should be wrapped up with the a single quote at the end. An example is shown below:
[root@amb171hawq /]# curl -u admin:admin -H 'X-Requested-By:admin' -X PUT  -d '{
  "AlertDefinition" : {
    "cluster_name" : "amb171hawq",
    "component_name" : "AMBARI_AGENT",
    "description" : "This host-level alert is triggered if the amount of disk space used on a host goes above specific thresholds. The default values are 60% for WARNING and 90% for CRITICAL.",
    "enabled" : true,
    "id" : 42,
    "ignore_host" : false,
    "interval" : 1,
    "label" : "Ambari Agent Disk Usage",
    "name" : "ambari_agent_disk_usage",
    "scope" : "HOST",
    "service_name" : "AMBARI",
    "source" : {
      "parameters" : [
        {
          "name" : "minimum.free.space",
          "display_name" : "Minimum Free Space",
          "units" : "bytes",
          "value" : 5.0E9,
          "description" : "The overall amount of free disk space left before an alert is triggered.",
          "type" : "NUMERIC",
          "threshold" : "WARNING"
        },
        {
          "name" : "percent.used.space.warning.threshold",
          "display_name" : "Warning",
          "units" : "%",
          "value" : 0.6,
          "description" : "The percent of disk space consumed before a warning is triggered.",
          "type" : "PERCENT",
          "threshold" : "WARNING"
        },
        {
          "name" : "percent.free.space.critical.threshold",
          "display_name" : "Critical",
          "units" : "%",
          "value" : 0.9,
          "description" : "The percent of disk space consumed before a critical alert is triggered.",
          "type" : "PERCENT",
          "threshold" : "CRITICAL"
        }
      ],
      "path" : "alert_disk_space.py",
      "type" : "SCRIPT"
    }
  }
}' http://amb171hawq:8080/api/v1/clusters/amb171hawq/alert_definitions/42

9. Check the alert in Ambari, the description should have changed.


Powered by Wolken Software