numpy datetime64 from unix utc seconds

Question

Note: I think datetime64 is doing the right thing. So I'll just leave the post up in case it's useful.

As of numpy 1.7.0, seconds passed in to a np.datetime64 are interpreted as being in a local timezone. Is there a clean and fast way to import a unix utc seconds to np.datetime64? I've got arrays with 50M of these and it seems that there should be a way to tell np.datetime64 that my seconds value is UTC, no?

datetime.datetime.utcfromtimestamp(1338624706)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # this is the time I'm looking for

np.datetime64(1338624706, 's')
numpy.datetime64('2012-06-02T01:11:46-0700')  # Darn you ISO!  Off by 7 hours

dt64 = np.datetime64(1338624706, 's')
dt64.astype(datetime.datetime)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # Wait, did it do the right thing?

# This seems like the best option at the moment,
# but requires building datetime.datetime objects:
dt64 = np.datetime64(datetime.datetime.utcfromtimestamp(1338624706))
numpy.datetime64('2012-06-02T01:11:46.000000-0700') # Show this
dt64.astype(datetime.datetime)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # Looks like it worked

I really do not want to resort to string operations. I would be nice to be able to convert an array of unix utc ints or floats straight to the correct dt64.

https://stackoverflow.com/a/13704307/417578 implies that numpy 1.8.0 might do what I want, but is there something that will work in 1.7.0?

Answer 1

Heres another way in pandas (which handles the quirks in different versions of numpy datetime64 correctly, so this works in numpy 1.6.2) - I think u might need current master for this (0.11-dev)

# obviously replace this by your utc seconds
# need to convert to the default in pandas of datetime64[ns]
z = pd.Series([(1338624706 + i)*1e9 for i in range(50)],dtype='datetime64[ns]')

In [35]: z.head()
Out[35]: 
0   2012-06-02 08:11:46
1   2012-06-02 08:11:47
2   2012-06-02 08:11:48
3   2012-06-02 08:11:49
4   2012-06-02 08:11:50
Dtype: datetime64[ns]

# turn it into a DatetimeIndex and localize
lidx = pd.DatetimeIndex(z).tz_localize('UTC')

<class 'pandas.tseries.index.DatetimeIndex'>
[2012-06-02 08:11:46, ..., 2012-06-02 08:12:35]
Length: 50, Freq: None, Timezone: UTC

# now you have a nice object to say convert timezones
In [44]: lidx.tz_convert('US/Eastern')
Out[44]: 
<class 'pandas.tseries.index.DatetimeIndex'>
[2012-06-02 04:11:46, ..., 2012-06-02 04:12:35]
Length: 50, Freq: None, Timezone: US/Eastern