Dropping metrics from the kubernetes-service-endpoints job in Prometheus

Question

I'm using the kube-prometheus-stack chart and I'm trying to get rid of all the kube_secret metrics.

When I query these metrics I see that they originate from a job named kubernetes-service-endpoints, but I can't figure out what service monitor controls it in values.yaml, so I tried dropping them from every single configurable service monitor - in each one of them I put:

    metricRelabelings:
    - action: drop
      regex: 'kube_secret_.+'
      sourceLabels: [__name__]

Didn't help of course. What do I need to define in values.yaml in order to drop those metrics? Thanks

Answer 1

I'm silly - I defined the kubernetes-service-endpoints job in prometheus.prometheusSpec.additionalScrapeConfigs...

In order to remove all kube_secret metrics I dropped metric names that matched kube_secret_.+ for this job and in the service monitor of kubeStateMetrics (kubeStateMetrics.serviceMonitor.metricRelabelings)