Precisely catch DNS error with Python requests

Question

I am trying to make a check for expired domain name with python-requests.

import requests

try:
    status = requests.head('http://wowsucherror')
except requests.ConnectionError as exc:
    print(exc)

This code looks too generic. It produces the following output:

HTTPConnectionPool(host='wowsucherror', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

What I'd like to do is to catch this DNS error only (like ERR_NAME_NOT_RESOLVED in Chrome). As a last resort I can just do string matching, but maybe there is a better, more structured and forward compatible way of dealing with this error?

Ideally it should be some DNSError extension to requests.

UPDATE: The error on Linux is different.

HTTPConnectionPool(host='wowsucherror', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',))

Reported bug to requests -> urllib3 https://github.com/shazow/urllib3/issues/1003

UPDATE2: OS X also reports different error.

requests.exceptions.ConnectionError: HTTPConnectionPool(host='wowsucherror', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))

Answer 1

Done this with this hack, but please monitor https://github.com/psf/requests/issues/3630 for a proper way to appear.

# for Python 2 compatibility
from __future__ import print_function
import requests

def sitecheck(url):
    status = None
    message = ''
    try:
        resp = requests.head('http://' + url)
        status = str(resp.status_code)
    if ("[Errno 11001] getaddrinfo failed" in str(exc) or     # Windows
        "[Errno -2] Name or service not known" in str(exc) or # Linux
        "[Errno 8] nodename nor servname " in str(exc)):      # OS X
        message = 'DNSLookupError'
    else:
        raise

    return url, status, message

print(sitecheck('wowsucherror'))
print(sitecheck('google.com'))

Answer 2

You could use lower-level network interface, socket.getaddrinfo https://docs.python.org/3/library/socket.html#socket.getaddrinfo

import socket


def dns_lookup(host):
    try:
        socket.getaddrinfo(host, 80)
    except socket.gaierror:
        return False
    return True


print(dns_lookup('wowsucherror'))
print(dns_lookup('google.com'))