First Value in Pandas DatetimeIndex is Unsearchable

Question

I have a simple Series:

>>> sub_dim_metrics
date
2017-04-04 00:00:00+00:00     32.38
2017-04-03 00:00:00+00:00    246.28
2017-04-02 00:00:00+00:00    146.25
2017-04-01 00:00:00+00:00    201.98
2017-03-31 00:00:00+00:00    274.74
2017-03-30 00:00:00+00:00    257.82
2017-03-29 00:00:00+00:00    279.38
2017-03-28 00:00:00+00:00    203.53
2017-03-27 00:00:00+00:00    250.65
2017-03-26 00:00:00+00:00    180.59
2017-03-25 00:00:00+00:00    196.61
2017-03-24 00:00:00+00:00    281.04
2017-03-23 00:00:00+00:00    276.44
2017-03-22 00:00:00+00:00    227.55
2017-03-21 00:00:00+00:00    267.59
Name: area, dtype: float64
>>> sub_dim_metrics.index
DatetimeIndex(['2017-04-04', '2017-04-03', '2017-04-02', '2017-04-01',
               '2017-03-31', '2017-03-30', '2017-03-29', '2017-03-28',
               '2017-03-27', '2017-03-26', '2017-03-25', '2017-03-24',
               '2017-03-23', '2017-03-22', '2017-03-21'],
              dtype='datetime64[ns, UTC]', name=u'date', freq=None)

Later in my code I retrieve the area for specific days using the following format: sub_dim_metrics['2017-04-02'], for example.

Before I retrieve area for a certain day, I first verify that the requested date is in the Series, like so: if '2017-04-02' in sub_dim_metrics.index

My problem is that the first value in the Index does not return true, while the rest do:

>>> '2017-04-02' in sub_dim_metrics.index
True
>>> '2017-04-04' in sub_dim_metrics.index
False

Why is this and what is the best way to verify a date is in my Series before retrieving its corresponding value?

Answer 1

IIUC:

You are getting False when you expect True:
You are checking whether a string is in a datetime index. Apparently pandas is loose with the check and tries to do it for you. It's getting it wrong though, isn't it.

plan 1
Do it right!

pd.to_datetime('2017-04-04') in sub_dim_metrics.index

True

plan 2
I think the unsorted-ness is throwing it off. sort_values first.

'2017-04-04' in sub_dim_metrics.index.sort_values()

True

---

setup

from io import StringIO
import pandas as pd

txt = """2017-04-04 00:00:00+00:00     32.38
2017-04-03 00:00:00+00:00    246.28
2017-04-02 00:00:00+00:00    146.25
2017-04-01 00:00:00+00:00    201.98
2017-03-31 00:00:00+00:00    274.74
2017-03-30 00:00:00+00:00    257.82
2017-03-29 00:00:00+00:00    279.38
2017-03-28 00:00:00+00:00    203.53
2017-03-27 00:00:00+00:00    250.65
2017-03-26 00:00:00+00:00    180.59
2017-03-25 00:00:00+00:00    196.61
2017-03-24 00:00:00+00:00    281.04
2017-03-23 00:00:00+00:00    276.44
2017-03-22 00:00:00+00:00    227.55
2017-03-21 00:00:00+00:00    267.59"""

sub_dim_metrics = pd.read_csv(StringIO(txt),
    sep='\s{2,}', engine='python',
    index_col=0, parse_dates=[0],
    header=None, names=['date', 'area'],
    squeeze=True)

Answer 2

  len(s.get_value('2017-04-02)) == 0
  False 

The key exists.

  len(s.get_value('2015-01-01)) == 0
  True

The key does not exists.