Applying set operation on list of dictionary

Question

My list contains same dictionaries multiple times like

Like

[ {'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}, {'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}]

When I apply set operation on this to make it distinct, it gives

TypeError: unhashable type: 'dict'

What is correct way to make this sort of list item distinct?

Answer 1

Dictionaries are mutable objects so can't be stored in a set. You could convert the dictionaries to an immutable object that represents the same data; after applying a set() operation to these you can then convert back to dictionaries:

unique = [dict(t) for t in set(tuple(sorted(d.items())) for d in l)]

This converts each dictionary to a sorted tuple containing all key-value pairs; this assumes that the values in the dictionary are immutable as well however.

Sorting is required because it is possible to produce two dictionaries that have the same key-value pairs but list these in a different order due to hash collisions; two keys can map to the same hash-table slot but the one that's inserted first will win:

>>> {'bar': 42, 'baz': 81} == {'baz': 81, 'bar': 42}
True
>>> list({'bar': 42, 'baz': 81}) == list({'baz': 81, 'bar': 42})  # compare keys in order
False

Answer 2

A straightforward way can be to convert each dict into string and use a set on it and then convert each string back to dict

>>> import ast
>>> l=[ {'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}, {'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}]
>>> l
[{'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}, {'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}]
>>> [ast.literal_eval(i) for i in set(map(str, l))]
[{'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}]
>>>

Another approach for different ordering where in we convert each dict to a tuple:

>>> l=[ {'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}, {'gram': '1', 'name': 'ZYLOG SYSTEMS LTD'}]
>>> set([tuple(d.items()) for d in l])
{(('name', 'ZYLOG SYSTEMS LTD'), ('gram', '1'))}
>>> [dict(i) for i in set([tuple(sorted(d.items())) for d in l])]
[{'name': 'ZYLOG SYSTEMS LTD', 'gram': '1'}]
>>>