Selectively blocking metrics originating from the Wavefront operator with Wavefront Proxy preprocessor rules
search cancel

Selectively blocking metrics originating from the Wavefront operator with Wavefront Proxy preprocessor rules

book

Article ID: 390480

calendar_today

Updated On:

Products

Observability

Issue/Introduction

This article demonstrates how to setup preprocessor rules with the Wavefront Proxy to selectively block metrics being generated from the Wavefront Operator itself.

Environment

Wavefront Operator

Resolution

To achieve your goal of disabling selected metrics from the Wavefront Operator, for example, certain control plane metrics, you need to add a custom resource file with preprocessor rules so that the proxy knows which metrics to reject/block.

Please refer to the following link for an example of preprocessor rules: Wavefront Proxy Preprocessor Rules YAML

Here’s an example of what two blocking rules could look like:

data:
  rules.yaml: |
    '2878':
      - rule    : block-controlplane-histogram
        action  : block
        scope   : metricName
        match   : "kubernetes.controlplane.workqueue.*"

      - rule    : block-controlplane-histogram1
        action  : block
        scope   : metricName
        match   : "kubernetes.controlplane.apiserver.*"
 
These rules will block all the metrics from kubernetes.controlplane.workqueue.* and kubernetes.controlplane.apiserver.* namespaces. That would include the following histogram metrics for example:
  • kubernetes.controlplane.apiserver.request.duration.seconds.m
  • kubernetes.controlplane.workqueue.queue.duration.seconds.m

Additional Information

You just need to add the metric names you want to block to the match sections in the rules. Once configured, you can check the logs to see histograms are being blocked by running the following command:

controlplane:~$ k logs -n observability-system wavefront-proxy-7b656cc684-wc85g | grep -i histogram | grep -i block

The logs should show entries like:

2025-03-06 17:19:53,262 INFO [AbstractReportableEntityHandler:printTotal] [2878] Histograms processed since start: 651; blocked: 82
2025-03-06 17:20:53,262 INFO [AbstractReportableEntityHandler:printTotal] [2878] Histograms processed since start: 725; blocked: 88
2025-03-06 17:21:53,262 INFO [AbstractReportableEntityHandler:printTotal] [2878] Histograms processed since start: 795; blocked: 101
2025-03-06 17:22:53,262 INFO [AbstractReportableEntityHandler:printTotal] [2878] Histograms processed since start: 863; blocked: 129