Implement hooks for list & dict changes

Question

My app relies on list and dict data structures for maintaining current state. Now I need to track whenever element is added/removed from list or dict data changes. I've found out that there are collections.abc.Sequence (for list) and collections.abc.MutableMapping (for dict), but they're very limited and result can't be used instead of list/dict (append, clear, ...).

I've been thinking about some proxy class that'll forward calls and provide hooks to be called before/after some method is forwarded, but didn't find anything that even looks like in.

How do I hook mutators of given structures? Is there something I don't know?

Answer 1

Even though subclassing built-in classes is officially supported, I'd still recommend you consider using a proxy class.

The problem is it isn't well documented when overriden methods are called. For instance, if you override __setitem__ method, it won't be called when you modify your dict using extend method, e.g. mydict.extend({"a": 42}). You'll have to override extend, too. It's very easy to forget to override some obscure method, and modification will occur silently without your callback getting called (whereas if you were using a proxy, it would throw an AttributeError indicating that you forgot to define some method).

There's even more to that. Some built-in methods that receive a dictionary through argument will not call any overriden methods at all in CPython, as shown in the following example from PyPy wiki:

>>>> class D(dict):
....     def __getitem__(self, key):
....         return 42
....
>>>>
>>>> d1 = {}
>>>> d2 = D(a='foo')
>>>> d1.update(d2)
>>>> print d1['a']
foo # but 42 in PyPy

Answer 2

Subclassing dict maybe:

class DictWatch(dict):
    def __init__(self, *args, **kwargs):
        self.callback = kwargs.pop('callback')
        dict.__init__(self, args)

    def __setitem__(self, key, val):
        # YOUR HOOK HERE
        self.callback(key, val)
        dict.__setitem__(self, key, val)

    # and other overrided dict methods if you need them

Demo:

>>> def cb(k,v):
...     print k,v
... 
>>> d=DictWatch(callback=cb)
>>> d['key']='100'
key 100