I am polling the same url several times in a row with urllib3. I have profiled the requests and I am getting the header:
Cache-Control: no-transform, max-age=120
On requests after the first I am getting the cached version of the page rather than the request being run again. I have no control over the headers returned from the server, how can I prevent the caching?
I am on Debian Squeeze and this is running python 2.6.6
pool = urllib3.HTTPConnectionPool('itunes.apple.com')
request = pool.request('GET', '/webObjects/MZStore.woa/wa/viewTop?
selected_tab_index=0&startIndex=0&genreId=36',
headers = {'Host': 'itunes.apple.com',
'Accept-Encoding': 'gzip, deflate', 'X-Apple-Tz': -18000,
'X-Apple-Store-Front': '143441-1,2')
urllib3 has no native caching built in. Are you sure it's not the server responding with cached results?
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With