Pandas groupby and make set of items

Question

I am using pandas groupby and want to apply the function to make a set from the items in the group.

The following results in TypeError: 'type' object is not iterable:

df = df.groupby('col1')['col2'].agg({'size': len, 'set': set})

But the following works:

def to_set(x):
    return set(x)
    
df = df.groupby('col1')['col2'].agg({'size': len, 'set': to_set})

In my understanding the two expression are similar, what is the reason why the first does not work?

Answer 1

Update

• As late as pandas version 0.22, this is an issue.
• As of pandas version 1.1.2, this is not an issue. Aggregating set, doesn't result in TypeError: 'type' object is not iterable.
  • Not certain when the functionality was updated.

Original Answer

It's because set is of type type whereas to_set is of type function:

type(set)
<class 'type'>

def to_set(x):
    return set(x)

type(to_set)

<class 'function'>

According to the docs, .agg() expects:

arg : function or dict

Function to use for aggregating groups.

• If a function, must either work when passed a DataFrame or when passed to DataFrame.apply.

• If passed a dict, the keys must be DataFrame column names.

Accepted Combinations are:

• string cythonized function name
• function

• list of functions

• dict of columns -> functions

• nested dict of names -> dicts of functions

Answer 2

Try using:

df = df.groupby('col1')['col2'].agg({'size': len, 'set': lambda x: set(x)})

Works for me.

Answer 3

Update for newer versions of Pandas if you get the following error

SpecificationError: nested renamer is not supported

df = df.groupby('col1')['col2'].agg(size= len, set= lambda x: set(x))