Django: is it possible to perform queries on a cached QuerySet instead of querying the database?

Question

I often need to cache results and perform ORM-like operations but with normal Python logic because I don't want the database to be hit due to performance issues.

For this very reason I wonder if Django offers a way of performing ORM operations on a cached QuerySet, instead of querying the database every single time the core query we are using changes slightly.

To visualize what I am saying, let's say we have these different queries:

# when evaluated it hits the db
devs = Worker.objects.filter(job__category=JobCategory.DEVELOPER)

# when evaluated it hits the db again
young_devs_salary = devs.filter(dob__gte=datetime.now() - relativedelta(years=24)).values('dob', 'salary')

# it would hit the db again
wellpaid_devs = devs.filter(salary__gte=high_salary_yearly)

Note that the first QuerySet would be sort of a superset of the rest of the following QuerySets.

What I am after is figuring out a way to tell Django that only the very first query, devs, should hit the db, and then cache its results, while the other queries, which use devs, should not query the database but the cached QuerySet instead.

Is this possible? And if Django does not support this, why? Maybe my scenario is not that common? But somewhat I doubt it, because I stumble upon similar situations daily.

Answer 1

As Daniel already answered, no you cannot use the querying API that way without finally hitting the database. Now you don't need anything special to filter a queryset's results in pure python - that's mostly standard list filtering using standard python features - but chances here it will actually not improve performances that much, if at all. Avoiding useless db hits is fine but that doesn't mean the database is your ennemy, and a decently tuned database with a sane schema will probably outperform pure python list filtering on most cases.

Also and FWIW if the Queryset API did support pure-python filtering it would be implemented as standard list filtering operations etc so it would buy you much, quite on the contrary.

As an example - the pure python version of:

young_devs_salary = devs.filter(dob__gte=datetime.now() - relativedelta(years=24)).values('dob', 'salary')

would be:

target_date = datetime.now() - relativedelta(years=24)
young_devs_salary = [(dev.dob, dev.salary) for dev in devs if dev.dob >= target_date]

As you can see there's no need for anything more in the Queryset api, but I doubt this will be much faster than querying the database unless you have a very small dataset or a very poor network connection between your django app and the database server (or any other infrastructure / database tuning issue).