Why do I get different results and types when accessing unmasked elements of a NumPy MaskedArray in different ways?

Question

I'm working with NumPy MaskedArray. I've noticed that when I try to access and perform calculations on the unmasked elements, using data[~data.mask] versus data.data[~data.mask] yields different results in terms of precision and the type of the resulting array.

Here's a minimal reproducible example:

import numpy as np

data = np.array([1.23, 4.56, 7.89], dtype=np.float32)
no_value = np.float32(-9999)
factor = 100
# dtype = np.int16 # This line is not directly relevant to the issue, so commented out

data = np.ma.masked_equal(data, no_value)

# 1. Using data[~data.mask]
result1 = data[~data.mask] * factor
print(result1, type(result1), result1.dtype)

# 2. Using data.data[~data.mask]
result2 = data.data[~data.mask] * factor
print(result2, type(result2), result2.dtype)

Running this code produces the following output:

[123.00000191 455.99999428 788.99998665] <class 'numpy.ma.core.MaskedArray'> float32
[123.         456.         789.        ] <class 'numpy.ndarray'> float32

The result of data[~data.mask] * factor remains a MaskedArray, and the values seem to include precision errors attributable to the original float32 dtype. On the other hand, the result of data.data[~data.mask] * factor is an ndarray, and the values are closer to what I would expect.

Why do these different access methods lead to differences in the type of the result (MaskedArray vs ndarray) and the precision of the floating-point arithmetic? Specifically, I'd like to understand why using data[~data.mask] returns a MaskedArray and why its internal calculation process might differ from directly referencing data.data.

I was hoping the results shown above would be the same

Answer 1

The result of data[~data.mask] * factor remains a MaskedArray, and the values seem to include precision errors attributable to the original float32 dtype.

To see what is happening, break up that expression. data[~data.mask] is a masked array with dtype float32. When that intermediate array is multiplied by the Python integer 100, the result is a masked array with dtype float64. (That's what I get with NumPy 1.26, 2.0, 2.1 and 2.2 on a Mac; I don't know why your output shows float32 for result1.dtype.) The type promotion that occurs when a masked array is multiplied by a Python integer is different from that of a regular array. For example, in the following, a is a regular array, and m is a masked array.

In [85]: a = np.array([1.23, 4.56, 7.89], dtype=np.float32)

In [86]: (a*100).dtype
Out[86]: dtype('float32')

In [87]: m = np.ma.masked_array(a)

In [88]: (m*100).dtype
Out[88]: dtype('float64')

There is an issue for this problem in the NumPy github repository.

A quick fix is to avoid the problematic type promotion by defining factor to be a float32:

factor = np.float32(100)