Google App Engine counts

Question

What is the appropriate way to handle counts in app engine(ndb or db)?

I have two projects on is a django-nonrel and the other is a pure django project but both need the ability to take a query and get a count back. The results could be greater than 1,000.

I saw some posts that said I could use Sharded Counters but they are counting all entities. I need to be able to know how many entities have the following properties x=1,y=True,z=3

#Is this the appropriate way?
count = some_entity.gql(query_string).count(SOME_LARGE_NUMBER)

Answer 1

The datastore is not good at this sort of query, because of tradeoffs to make it distributed. These include fairly slow reads, and very limited indexing.

If there are a limited set of statistics you need (number of users, articles, etc) then you can keep running totals in a separate entity. This means you need to do two writes(puts) when something changes: one for the entity that changes, and one to update the stats entity. But you only need one read(get) to get your statistics, instead of however many entities they are distilled from.

You may be uncomfortable with this because it goes against what we all learned about normalisation, but it is far more efficient and in many cases works fine. You can always have a cron job periodically do your queries to check the statistics are accurate, if this is critical.