Probes returning zero

Question

I've been struggling to understand why some HTTP endpoints return '0' for 'probe_success' and 'probe_http_status_code', while being perfectly able to get a "valid" response with curl.

Example: curl -s "localhost:9115/probe?target=http://linux.org&module=http_2xx" | grep -v '^#'

Output:

probe_dns_lookup_time_seconds 0.003712821
probe_duration_seconds 0.212811871
probe_failed_due_to_regex 0
probe_http_content_length 0
probe_http_duration_seconds{phase="connect"} 0.002263513
probe_http_duration_seconds{phase="processing"} 0.196389853
probe_http_duration_seconds{phase="resolve"} 0.006723945
probe_http_duration_seconds{phase="tls"} 0
probe_http_duration_seconds{phase="transfer"} 2.6001e-05
probe_http_redirects 1
probe_http_ssl 0
probe_http_status_code 0
probe_http_version 0
probe_ip_protocol 4
probe_success 0

Here is the job definition:

  - job_name: 'blackbox'
    scrape_interval: 30s
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets:
        - http://linux.org
    relabel_configs:
      - source_labels: [__address__]
        regex: '(.*)(:80)?'
        target_label: __param_target
      - source_labels: [__param_target]
        regex: '(.*)'
        target_label: instance
        replacement: '${1}'
      - source_labels: []
        regex: '.*'
        target_label: __address__
        replacement: 'blackbox:9115'

and the module definition:

modules:
  http_2xx:
    prober: http
    timeout: 15s
    http:
      valid_status_codes: []
      method: GET

Answer 1

If you append &debug=true you'll get some details on the error.

My guess is blackbox_exporter cannot connect because it defaults to ipv6 connections and that's not working.

In my case (same problem), the debug output says

level=error msg="Resolving target address" ip_protocol=ip6
level=error msg="Resolution with IP protocol failed (fallback_ip_protocol is false): err"
level=error msg="Error resolving address" err="address apple.com: no suitable address found"
level=error msg="Probe failed" duration_seconds=0.003648031

From the blackbox documentation:

  # The IP protocol of the HTTP probe (ip4, ip6).
  [ preferred_ip_protocol: <string> | default = "ip6" ]
  [ ip_protocol_fallback: <boolean> | default = true ]

So, if you modify your module configuration and restart blackbox exporter, it should work:

modules:
  http_2xx:
    prober: http
    timeout: 15s
    http:
      valid_status_codes: []
      method: GET
      preferred_ip_protocol: "ip4" # <---- !

Answer 2

In my error was: "x509: certificate signed by unknown authority"

Change option in http section

 tls_config:
    insecure_skip_verify: true