Do something on every Celery Task run

Question

Assume I have some Celery tasks which are based on one abstract task.

class BaseTask(Task):
    abstract = True

    def on_failure(self, exc, task_id, args, kwargs, einfo):
        logging.info('Task failed')

    def on_success(self, retval, task_id, args, kwargs):
        logging.info('Task success')

    # NO SUCH METHOD IN TASK
    # def on_start(self):
    #     do_something_on_every_task_start()

@app.task(base=BaseTask)
def task1(x):
    print x

@app.task(base=BaseTask)
def task2(y):
    print y

I want to do something on start of every task execution. Are there any possibilities to achieve this result? Thanks.

Answer 1

You can override the __call__ method. This will work both if you call the task synchronously with task1(1) or call it asynchronously, with task1.delay(1). In the following example, you'll get "CALLED!" on the console or in the logs depending on how you call the task and how your logging is setup.

class BaseTask(Task):
    abstract = True

    def __call__(self, *args, **kwargs):
        print "CALLED!"
        return super(BaseTask, self).__call__(*args, **kwargs)

@app.task(base=BaseTask)
def task1(x):
    print x

@app.task(base=BaseTask)
def task2(y):
    print y

Answer 2

If need 'semantically beautiful' method, you can try signals.

http://docs.celeryproject.org/en/latest/userguide/signals.html

Here is another useful post on pre/post task callbacks.

https://gist.github.com/nuria/35f332acfb84ecf80e3b