Go to next item in list on error

Question

I am pulling websites from a list and want to test, whether they are up or down. The code below works fine as long as they are up, but as soon as something is wrong with one of these urls, I get an error message and the whole scrip stops.

What I want to achieve: Error message == website not working therefore print down and move to next item in list.

import urllib2
    from urllib2 import Request, urlopen, HTTPError, URLError

def checkurl(z):
    user_agent = 'Mozilla/20.0.1 (compatible; MSIE 5.5; Windows NT)'
    headers = { 'User-Agent':user_agent }
    link = "http://"+z
    req = Request(link, headers = headers)
    try:
        page_open = urlopen(req)
    except HTTPError, e:
        print "down"
    else:
        print 'up'
    #print urllib2.urlopen('http://'+z).read()


Traceback (most recent call last):
  File "/home/user/Videos/python/onion/qweqweqweq.py", line 48, in <module>
    checkurl(x)
  File "/home/user/Videos/python/onion/qweqweqweq.py", line 23, in checkurl
    page_open = urlopen(req)
  File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 401, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 419, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1211, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1178, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 962, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 996, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 958, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 818, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 780, in send
    self.connect()
  File "/usr/lib/python2.7/httplib.py", line 761, in connect
    self.timeout, self.source_address)
  File "/home/user/Videos/python/onion/qweqweqweq.py", line 5, in create_connection
    sock.connect(address)
  File "/usr/lib/python2.7/dist-packages/socks.py", line 369, in connect
    self.__negotiatesocks5(destpair[0],destpair[1])
  File "/usr/lib/python2.7/dist-packages/socks.py", line 236, in __negotiatesocks5
    raise Socks5Error(ord(resp[1]),_generalerrors[ord(resp[1])])
TypeError: __init__() takes exactly 2 arguments (3 given)

Answer 1

You are catching HTTPError, but what is thrown is Socks5Error.

Answer 2

You're missing Socks5Error in your except clause. Look at the traceback:

raise Socks5Error(ord(resp[1]),_generalerrors[ord(resp[1])])

Note that this wouldn't have happened if you used requests instead of urllib2. The interface is a lot clearer, the documentation better.