Django related objects are missing from celery task (race condition?)

Question

Strange behavior, that I don't know how to explain. I've got a model, Track, with some related points. I call a celery task to performs some calculations with points, and they seem to be perfectly reachable in the method itself, but unavailable in celery task.

@shared_task
def my_task(track):
    print 'in the task', track.id, track.points.all().count()

def some_method():
    t = Track()
    t.save()
    t = fill_with_points(t)  # creating points, attaching them to a Track
    t.save()
    print 'before the task', track.id, track.points.all().count()
    my_task.delay(t)

That prints the following:

before the task, 21346, 2971
in the task, 21346, 0

Strange thing though, when I put a time.sleep(10) at the first line of my_task or before calling my_task at all, it works out well, like there's some race condition. But the first printed line clearly says that points are available in the database, when it makes a select query (track.points.all().count()).

Answer 1

I'm going to assume this is due to transaction isolation.

Django transactions by default are tied to requests; and while a transaction is active, no other process will see the changes until the transaction is committed. If you're in the middle of a save method, and there are quite a lot of other actions that take place before the request finishes, it seems likely that Celery starts processing the task before the transaction is committed. You could fix this by committing manually or by delaying the task.