Numpy/Pandas clean way to check if a specific value is NaN

Question

How can I check if a given value is NaN?

e.g. if (a == np.NaN) (doesn't work)

Please note that:

• Numpy's isnan method throws errors with data types like string
• Pandas docs only provide methods to drop rows containing NaNs, or ways to check if/when DataFrame contains NaNs. I'm asking about checking if a specific value is NaN.
• Relevant Stackoverflow questions and Google search results seem to be about checking "if any value is NaN" or "which values in a DataFrame"

There must be a clean way to check if a given value is NaN?

Answer 1

You can use the inate property that NaN != NaN

so a == a will return False if a is NaN

This will work even for strings

Example:

In[52]:
s = pd.Series([1, np.NaN, '', 1.0])
s

Out[52]: 
0      1
1    NaN
2       
3      1
dtype: object


for val in s:
    print(val==val)
True
False
True
True

This can be done in a vectorised manner:

In[54]:
s==s

Out[54]: 
0     True
1    False
2     True
3     True
dtype: bool

but you can still use the method isnull on the whole series:

In[55]:
s.isnull()

Out[55]: 
0    False
1     True
2    False
3    False
dtype: bool

UPDATE

As noted by @piRSquared if you compare None==None this will return True but pd.isnull will return True so depending on whether you want to treat None as NaN you can still use == for comparison or pd.isnull if you want to treat None as NaN

Answer 2

Pandas has isnull, notnull, isna, and notna

These functions work for arrays or scalars.

---

Setup

a = np.array([[1, np.nan],
              [None, '2']])

---

Pandas functions

pd.isna(a)
# same as
# pd.isnull(a)

array([[False,  True],
       [ True, False]])

---

pd.notnull(a)
# same as
# pd.notna(a)

array([[ True, False],
       [False,  True]])

---

DataFrame (or Series) methods

b = pd.DataFrame(a)

---

b.isnull()
# same as
# b.isna()

       0      1
0  False   True
1   True  False

---

b.notna()
# same as
# b.notnull()

       0      1
0   True  False
1  False   True