Asynchronous feedparser requests

Question

I'm using feedparser (Python) to get some RSS entries from several websites.

How can I do asynchronous requests using feedparser? I mean, I want to get some RSS entries but I don't want to wait for the response. A callback function should be called when I get the response from the feedparser request. After the request (and probably before the reply) I want to do some computation.

Thank you all, Hugo

Answer 1

You're probably better off to decouple the fetching from the parsing. Feedparser is an amazing parsing library, but probably not the best HTTP client libary. Luckily that's fairly easy to do as Feedparser can also parse a blob of text.

Then, this means you can pick any HTTP library to actually do the polling, as long as it supports your asyncrhonous requirement. You'll probably end up using something like Twisted and its WebClient.

Another solution is to of course avoid doing all that expensive polling yourself and rely on a solution like Superfeedr which will use webhooks to send you only what's new in a given feed.