Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Initializing numpy masked array from Python int list with None values

As shown in the answer to the question Convert python list with None values to numpy array with nan values, it is straightforward to initialize a masked numpy array from a list with None values if we enforce the dtype=float. Those float values get converted to nan and we can simply do:

ma.masked_invalid(np.array(a, dtype=float), copy=False)

This however will not work for int like:

ma.masked_invalid(np.array(a, dtype=int), copy=False)

since the intermediate np.array will not be created with None values (there is no int nan).

What is the most efficient way to initialize a masked array based on Python list of ints that also contains None values in such way that those None values become masked?

like image 740
Andrzej Pronobis Avatar asked Oct 16 '25 18:10

Andrzej Pronobis


1 Answers

The most elegant solution I have found so far (and it is not elegant at all) is to initialize a masked array of type float and convert it to int afterwards:

ma.masked_invalid(np.array(a, dtype=float), copy=False).astype(int)

This generates a proper NP array where None values in the initial array a are masked. For instance, for:

a = [1, 2, 3, None, 4]
ma.masked_invalid(np.array(a, dtype=float), copy=False).astype(int)

we get:

masked_array(data = [1 2 3 -- 4],
             mask = [False False False  True False],
       fill_value = 999999)

Also, the actual masked int values become min int, i.e.

ma.masked_invalid(np.array(column, dtype=float), copy=False).astype(int).data

gives:

array([                   1,                    2,                    3,
       -9223372036854775808,                    4])
like image 165
Andrzej Pronobis Avatar answered Oct 19 '25 10:10

Andrzej Pronobis