Have Celery broadcast return results from all workers

Question

Is there a way to get all the results from every worker on a Celery Broadcast task? I would like to monitor if everything went ok on all the workers. A list of workers that the task was send to would also be appreciated.

Answer 1

No, that is not easily possible.

But you don't have to limit yourself to the built-in amqp result backend, you can send your own results using Kombu (http://kombu.readthedocs.org), which is the messaging library used by Celery:

from celery import Celery
from kombu import Exchange

results_exchange = Exchange('myres', type='fanout')

app = Celery()

@app.task(ignore_result=True)
def something():
    res = do_something()
    with app.producer_or_acquire(block=True) as producer:
        producer.send(
            {'result': res},
            exchange=results_exchange,
            serializer='json',
            declare=[results_exchange],
        )

producer_or_acquire will create a new kombu.Producer using the celery broker connection pool.