Heroku process exits with status 0 but its dyno still "crashes"

Question

I have a scrapping script written in ruby running on heroku inside a worker dyno. Even though the script runs smoothly and exits with status 0, heroku still tells me that the dyno crashed:

2016-12-13T00:59:10.695566+00:00 heroku[spider.1]: State changed from up to crashed
2016-12-13T00:59:10.683166+00:00 heroku[spider.1]: Process exited with status 0

What can I do to make the dyno stop correctly when the script ends?

Thanks.

Answer 1

Dynos that run as a part of your dyno formation are expected to be running continuously. The architecture that should work for you is:

1. Add the scraper in form of a command (e.g. a script with the executable bit set) that you'll be able to run via heroku run, e.g. heroku run -r production scrape.
2. Add a script that runs the command from point 1, e.g. you can use the platform API or heroku run -r production scrape.
3. Call the script in point 2 from the Heroku scheduler.

This should be a good starting point. There are some corner cases you need to handle like ensuring that only one scrape can be run at a time (if that's your requirement of course).

Answer 2

Sounds like a job for Heroku Scheduler and one-off dynos:

Heroku Scheduler executes scheduled jobs via one-off dynos

There’re 2 types: one-off and “dyno formation” https://devcenter.heroku.com/articles/one-off-dynos:

The set of dynos declared in your Procfile and managed by the dyno manager via heroku ps:scale are known as the dyno formation. These dynos do the app’s regular business (such as handling web requests and processing background jobs) as it runs. When you wish to do one-off administrative or maintenance tasks for the app, or execute some task periodically using Heroku Scheduler, you can spin up a one-off dyno.

There are four differences between one-off dynos (run with heroku run) and formation dynos (run with heroku ps:scale), one of them being restart policy:

One-off dynos never automatically restart, whether the process ends on its own or whether you manually disconnect.

This sounds like exactly what you need.