rolling function in python ignoring nans

Question

https://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.rolling_quantile.html

I cant not see how to best ignore NaNs in the rolling percentile function. Would anyone know?

seriestest = pd.Series([1, 5, 7, 2, 4, 6, 9, 3, 8, 10])

and insert nans

seriestest2 = pd.Series([1, 5, np.NaN, 2, 4, np.nan, 9, 3, 8, 10])

Now, on the first series, I get expected output, using:

seriestest.rolling(window = 3).quantile(.5)

But, I wish to do the same and ignore NaNs on the test2 series.

seriestest2.rolling(window = 3).quantile(.5)

Gives:

0    NaN
1    NaN
2    NaN
3    NaN
4    NaN
5    NaN
6    NaN
7    NaN
8    8.0
9    8.0
dtype: float64

But I think it gives something like this if we can parse a skipna=True, which doesn't work for me:

0    NaN
1    NaN
2    5.0
3    2.0
4    4.0
5    4.0
6    4.0
7    3.0
8    8.0
9    8.0
dtype: float64

Answer 1

The issue is that having nan values will give you less than the required number of elements (3) in your rolling window. You can define the minimum number of valid observations with rolling to be less by setting the min_periods parameter.

seriestest2.rolling(window=3, min_periods=1).quantile(.5)

Alternatively, if you simply want to replace nan values, with say 0, you can use fillna:

seriestest2.fillna(value=0).rolling(window=3).quantile(.5)