I want to divide one dataframe by another in Pandas to eventually represent a percentage change. Both dataframes values contain NaN and 0. Now, when I divide one dataframe by the other, the result where the value from both dataframes was zero is NaN. I know why 0/0 is set to np.nan, but from a percentage change perspective, I need 0/0 to be 0.
What is the cleanest way to accomplish this?
Reproducing the problem:
import pandas as pd
import numpy as np
data_with_zeros = pd.DataFrame({'a': [2, np.nan, 0, 3], 'b': [np.nan, 2, 0, 6]})
data_with_zeros['a'].div(data_with_zeros['b'], fill_value=0)
Result:
0 inf
1 0.0
2 NaN
3 0.5
dtype: float64
Here's an approach with dataframe.where method -
mask = (data_with_zeros[['a','b']].values == [0,0]).all(1)
data_with_zeros['a'].div(data_with_zeros['b'], fill_value=0).where(~mask,0)
Alternatively mask could be created in a bit more intuitive way, like so -
mask = (data_with_zeros.a == 0) & (data_with_zeros.b == 0)
Sample runs -
Case #1:
In [66]: data_with_zeros
Out[66]:
a b
0 2.0 NaN
1 NaN 2.0
2 0.0 0.0
3 3.0 6.0
In [67]: mask = (data_with_zeros.a == 0) & (data_with_zeros.b == 0)
In [68]: data_with_zeros['a'].div(data_with_zeros['b'], fill_value=0).where(~mask,0)
Out[68]:
0 inf
1 0.000000
2 0.000000
3 0.500000
dtype: float64
Case #2:
In [70]: data_with_zeros
Out[70]:
a b
0 2.0 0.0
1 NaN 2.0
2 0.0 0.0
3 3.0 6.0
In [71]: mask = (data_with_zeros.a == 0) & (data_with_zeros.b == 0)
In [72]: data_with_zeros['a'].div(data_with_zeros['b'], fill_value=0).where(~mask,0)
Out[72]:
0 inf
1 0.000000
2 0.000000
3 0.500000
dtype: float64
In addition to the solution provided by Divakar, this function uses that method to do a dataframe/dataframe division:
def divide(a, other, fill_value=None):
serie_name = a.name
mask = ((a == 0) & (other[serie_name] == 0))
result_with_zeros = a.div(other[serie_name], fill_value=fill_value)
result_filled = result_with_zeros.where(~mask,0)
return result_filled
data_with_zeros.apply(divide, args=(data_with_zeros,))
Result:
a b
0 1.0 0.0
1 NaN 1.0
2 0.0 0.0
3 1.0 1.0
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With