Experience with web crawlers on heroku

Question

Does anybody have experience coding web crawlers with gems such as anemone and deploying them to heroku for your own person use? Would such a continuously running programs violate any of heroku's TOA/TOS?

Answer 1

Not any more.

Heroku Acceptable Use Policy states in Prohibited Actions p.21 that crawler must

• identify itself via a unique User Agent
• obey robots.txt (including crawl-delay directive)
• from p.20 stems the requirement not use you crawler as an "open proxy"

NB! A free instance must not exceed 18 hours of work a day.