Why is a Django migration using the same random default on every row?

Question

Note:

I understand and am well aware of the difference between passing a function as a parameter and invoking a function and passing the result as a parameter. I believe I am passing the function correctly.

Specs

• Django 1.11
• PostgreSQL 10.4

Scenario:

I have dozens of models in my application, with many existing records. I need to add a random seed to each of these models that will get created and set when a new model instance is created. I also want to generate the random seed for the existing instances.

My understanding of how Django model defaults and Migrations work is that when a new field is added to a model, if that field has a default value set, Djano will update all existing instances with the new field and corresponding default.

However, despite the fact that I'm definitely passing a function as the default, and the function produces a random number, Django is using the same random number when updating existing rows (e.g. it seems that Djano is only calling the function once, then using the return value for all entries).

Example

A shortened version of my code:

def get_random():
    return str(random.random())

class Response(models.Model):
    # various properties
    random = models.CharField(max_length=40, default=get_random)
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    content = JSONField(null=True)

The random field is being added after the model and many instances of it have already been created. A makemigrations command appears to generate the proper migration file, with a migrations.AddField() call, passing in default=get_random as a parameter.

However, after running makemigrations, all existing existing Response instances contain the exact same number in their random field. Creating and saving new instances of the model work as expected (with a pseudo-unique random number).

Workaround

An easy workaround is to just run a one-time script that does a

for r in Response.objects.all():
    r.random = get_random()
    r.save()

Or override the model's save() method and then do a mass save. But I don't think these workarounds should be necessary. It also means that if I want to make a unique field with a random default, then I will need multiple migrations. First I would have to add the field with the assigned default. Next I would need to apply the migration and manually re-initialize the field values. Then a second migration to add the unique=True property.

It seems that if Django is to apply default values to existing instances upon a makemigrations then it should apply them using the same semantics as creating a new instance. Is there any way to force Django to call the function for each model instance when migrating?

Answer 1

To add a non-null column to an existing table, Django needs to use an ALTER TABLE ADD COLUMN ... DEFAULT <default_value>. This only allows Django to call the default function once, that's why you see every row having the same value.

Your workaround is pretty much spot on, except that you can populate the existing rows with unique values using a data migration, so that it doesn't require any manual steps. The entire procedure for this use-case is described in the docs: https://docs.djangoproject.com/en/2.1/howto/writing-migrations/#migrations-that-add-unique-fields