Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pivot table based on groupby in Pandas

I have a dataframe like this:

customer_id | date     | category
1           | 2017-2-1 | toys
2           | 2017-2-1 | food
1           | 2017-2-1 | drinks
3           | 2017-2-2 | computer
2           | 2017-2-1 | toys
1           | 2017-3-1 | food

>>> import pandas as pd
>>> dt = dict(customer_id=[1,2,1,3,2,1],
              date='2017-2-1 2017-2-1 2017-2-1 2017-2-2 2017-2-1 2017-3-1'.split(),
              category=["toys", "food", "drinks", "computer", "toys", "food"])) 
>>> df = pd.DataFrame(dt)

ues my new columns and one hot encoding those columns, I know I can use df.pivot_table(index = ['customer_id'], columns = ['category']).

>>> df['Indicator'] = 1 
>>> df.pivot_table(index=['customer_id'], columns=['category'],
                   values='Indicator').fillna(0).astype(int)                                                             
category     computer  drinks  food  toys
customer_id                              
1                   0       1     1     1
2                   0       0     1     1
3                   1       0     0     0
>>>  

I also want to group by date so each row only contains information from the same date, like in the desired output below, id 1 has two rows because two unique dates in the date column.

customer_id | toys | food | drinks | computer 
1           | 1    | 0    | 1      | 0        
1           | 0    | 1    | 0      | 0
2           | 1    | 1    | 0      | 0
3           | 0    | 0    | 0      | 1
like image 402
efsee Avatar asked Sep 08 '25 11:09

efsee


2 Answers

You may looking for crosstab

>>> pd.crosstab([df.customer_id,df.date], df.category)                                                                                                                
category              computer  drinks  food  toys
customer_id date                                  
1           2017-2-1         0       1     0     1
            2017-3-1         0       0     1     0
2           2017-2-1         0       0     1     1
3           2017-2-2         1       0     0     0
>>>
>>> pd.crosstab([df.customer_id,df.date],
                df.category).reset_index(level=1)                                                                                           
category         date  computer  drinks  food  toys
customer_id                                        
1            2017-2-1         0       1     0     1
1            2017-3-1         0       0     1     0
2            2017-2-1         0       0     1     1
3            2017-2-2         1       0     0     0
>>>
>>> pd.crosstab([df.customer_id, df.date], 
                df.category).reset_index(level=1, drop=True)                                                                                
category     computer  drinks  food  toys
customer_id                              
1                   0       1     0     1
1                   0       0     1     0
2                   0       0     1     1
3                   1       0     0     0
>>>   
like image 163
BENY Avatar answered Sep 10 '25 06:09

BENY


Assuming your frame is called df, you could add an indicator column and then directly use .pivot_table:

df['Indicator'] = 1

pvt = df.pivot_table(index=['date', 'customer_id'],
                     columns='category',
                     values='Indicator')\
        .fillna(0)

This gives a dataframe that looks like:

category              computer  drinks  food  toys
date     customer_id                              
2017-2-1 1                 0.0     1.0   0.0   1.0
         2                 0.0     0.0   1.0   1.0
2017-2-2 3                 1.0     0.0   0.0   0.0
2017-3-1 1                 0.0     0.0   1.0   0.0
like image 30
asongtoruin Avatar answered Sep 10 '25 07:09

asongtoruin