relabel_configs - drop specific metrics that have specific values for a label

Question

I have a metric istio_requests_total

I want to drop all the data fromistio_requests_total, which have specific label values like

istio_requests_total {reporter="source"}

I have tried metric relabel configs, but they apply to all metrics and not just istio_requests_total

  metric_relabel_configs:
  - source_labels: [reporter]
    regex: '^source$'
    action: drop

Answer 1

The following relabeling config must drop metrics matching the istio_requests_total{reporter="source"} series selector:

metric_relabel_configs:
- source_labels: [__name__, reporter]
  regex: 'istio_requests_total;source'
  action: drop

This relabeling rule works in the following way per each scraped metric:

1. It joins the metric name with the reporter label value. It uses ; separator for joining. The default separator can be changed if needed via separator option in the relabel config.
2. It matches the result from step 1 against the provided regex. The regex is automatically anchored to the beginning and the end of the matching string, so there is no need in specifying ^ and $ anchors in the regex.
3. If the regex matches the result from step 1, then the metric is dropped. Otherwise it isn't dropped.

P.S. I work on a Prometheus-like monitoring solution - VictoriaMetrics, which provides some improvements over Prometheus relabeling. These improvements can simplify some relabeling tasks as this one. For example, the following VictoriaMetrics-specific relabeling rule is equivalent to the rule above, but it looks more clear:

metric_relabel_configs:
- if: 'istio_requests_total{reporter="source"}'
  action: drop