pandas.isnull() not working on decimal type?

Question

do I miss something or do we have an issue with pandas.isnull() ?

>>> import pandas as pd
>>> import decimal
>>> d = decimal.Decimal('NaN')
>>> d
Decimal('NaN')
>>> pd.isnull(d)
False
>>> f = float('NaN')
>>> f
nan
>>> pd.isnull(f)
True
>>> pd.isnull(float(d))
True

Problem is I have a dataframe with decimal.Decimal values in it, and df.dropna() doesn't remove NaN for this reason...

Answer 1

Yes this isn't supported, you can use the property that NaN does not equal itself which still works for Decimal types:

In [20]:
import pandas as pd
import decimal
d = decimal.Decimal('NaN')
df = pd.DataFrame({'a':[d]})
df

Out[20]:
     a
0  NaN

In [21]:    
df['a'].apply(lambda x: x != x)

Out[21]:
0    True
Name: a, dtype: bool

So you can do:

In [26]:
df = pd.DataFrame({'a':[d,1,2,3]})
df[df['a'].apply(lambda x: x == x)]

Out[26]:
   a
1  1
2  2
3  3