Timeout within session while sending requests

Question

I'm trying to learn how I can use timeout within a session while sending requests. The way I've tried below can fetch the content of a webpage but I'm not sure this is the right way as I could not find the usage of timeout in this documentation.

import requests

link = "https://stackoverflow.com/questions/tagged/web-scraping"

with requests.Session() as s:
    r = s.get(link,timeout=5)
    print(r.text)

How can I use timeout within session?

Answer 1

According to the Documentation - Quick Start.

You can tell Requests to stop waiting for a response after a given number of seconds with the timeout parameter. Nearly all production code should use this parameter in nearly all requests.

requests.get('https://github.com/', timeout=0.001)

Or from the Documentation Advanced Usage you can set 2 values (connect and read timeout)

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

r = requests.get('https://github.com', timeout=(3.05, 27))

Making Session Wide Timeout

Searched throughout the documentation and it seams it is not possible to set timeout parameter session wide.

But there is a GitHub Issue Opened (Consider making Timeout option required or have a default) which provides a workaround as an HTTPAdapter you can use like this:

import requests
from requests.adapters import HTTPAdapter

class TimeoutHTTPAdapter(HTTPAdapter):
    def __init__(self, *args, **kwargs):
        if "timeout" in kwargs:
            self.timeout = kwargs["timeout"]
            del kwargs["timeout"]
        super().__init__(*args, **kwargs)

    def send(self, request, **kwargs):
        timeout = kwargs.get("timeout")
        if timeout is None and hasattr(self, 'timeout'):
            kwargs["timeout"] = self.timeout
        return super().send(request, **kwargs)

And mount on a requests.Session()

s = requests.Session() 
s.mount('http://', TimeoutHTTPAdapter(timeout=5)) # 5 seconds
s.mount('https://', TimeoutHTTPAdapter(timeout=5))
...
r = s.get(link) 
print(r.text)

or similarly you can use the proposed EnhancedSession by @GordonAitchJay

with EnhancedSession(5) as s: # 5 seconds
    r = s.get(link)
    print(r.text)