Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to map NA values of the type pandas._libs.missing.NAType

Tags:

python

pandas

nan

Somewhere along my workflow NaN values in a Pandas DataFrame (filled in using np.Nan) have turned into <NA> values. (I am still trying to figure out how this happened. Reimporting the dataset from a CSV might be responsible?) pandas.DataFrame.dropna works fine. However pandas.DataFrame.isna only maps

NA values, such as None or numpy.NaN [...] Everything else gets mapped to False values.

Is there a way to map NA values of the type pandas._libs.missing.NAType?

fictitious sample

In [1]: import numpy as np
        import pandas as pd

        dictionary = {'environment': ['test', 'prod', 'test', 'prod'], 
                      'event': ['add_rd', 'add_rd', 'add_env', 'add_env'], 
                      'entry': ['yes', np.NaN, 'no', np.NaN]
                     }

        df = pd.DataFrame(dictionary, columns= ['environment', 'event', 'entry'])

(something happes that turns NaN values into <NA> values of the type pandas._libs.missing.NAType)

In [3]: print(df)

          environment    event entry
        0        test   add_rd   yes
        1        prod   add_rd   <NA>
        2        test  add_env    no
        3        prod  add_env   <NA>


Expected output:

In [4]: df["entry"].isna()

Out[4]  0    False
        1     True
        2    False
        3     True
        Name: entry, dtype: bool
like image 773
marianoju Avatar asked Oct 15 '25 20:10

marianoju


1 Answers

You can try pd.NA if you're pandas is updated:

df["isna"] = df["entry"].apply(lambda x:True if x is pd.NA else False)
like image 188
Arya Sadeghi Avatar answered Oct 18 '25 11:10

Arya Sadeghi