I have two columns in my dataframe - Start date and Finish date (It is a timestamp ). I would like to calculate runtime by finding the difference. When I run the code below, I get the error: TypeError: cannot subtract DatetimeArray from ndarray. I am also unable to split the finished date column into date and time so I can calculate runtime in another manner.
# calculate runtime
pd.to_datetime(df.start_date)
pd.to_datetime(df.finish_date)
diff = pd.to_datetime(df.finish_date) - pd.to_datetime(df.start_date)
This is how the dataframe looks like: df
file_name unique_id start_date finish_date
file1 1716b94a8d8d83c8fffe4bdd14d536ae1ee9cba6bf17e6... 2020-09-03T16:18:38.929863799Z 2020-09-03T16:20:17.615093582Z
file2 87ff84ab119b798312230fceb3a8730fe74669a373650a... 2020-09-03T16:26:25.075167073Z 2020-09-04T00:04:39.702686798Z
How can I -
Assume you want to calculate the number of days between the dates, then this is one solution:
import datetime as dt
diff = (pd.to_datetime(df.finish_date) - pd.to_datetime(df.start_date)).dt.days
EDIT
Another alternative is
Start = pd.to_datetime(df.finish_date)
End = pd.to_datetime(df.start_date)
End.subtract(Start)
Example: Here I choose to compute the difference between dates in a df and now.
metric_id device_id timestamp cpu_5min vol_max
0 device_1 2020-12-04 05:15:00 116.0 734.0
1 device_1 2020-12-04 05:30:00 213.0 325.0
2 device_1 2020-12-04 05:35:00 427.0 668.0
3 device_2 2020-12-04 05:15:00 540.0 NaN
4 device_2 2020-12-04 05:30:00 127.0 NaN
5 device_2 2020-12-04 05:35:00 654.0 NaN
and
df['tDATE'] = pd.to_datetime(df['timestamp'])
df['DIFF1'] = (df['tDATE'] - dt.datetime.now()).dt.days #method 11
df['DIFF2'] = df['tDATE'].subtract(dt.datetime.now()) #method2
which returns
metric_id device_id timestamp cpu_5min vol_max \
0 device_1 2020-12-04 05:15:00 116.0 734.0
1 device_1 2020-12-04 05:30:00 213.0 325.0
2 device_1 2020-12-04 05:35:00 427.0 668.0
3 device_2 2020-12-04 05:15:00 540.0 NaN
4 device_2 2020-12-04 05:30:00 127.0 NaN
5 device_2 2020-12-04 05:35:00 654.0 NaN
metric_id tDATE difd DIFF1 DIFF2
0 2020-12-04 05:15:00 -14 -14 -14 days +22:13:26.627607
1 2020-12-04 05:30:00 -14 -14 -14 days +22:28:26.627607
2 2020-12-04 05:35:00 -14 -14 -14 days +22:33:26.627607
3 2020-12-04 05:15:00 -14 -14 -14 days +22:13:26.627607
4 2020-12-04 05:30:00 -14 -14 -14 days +22:28:26.627607
5 2020-12-04 05:35:00 -14 -14 -14 days +22:33:26.627607
EDIT: Working with Timestamps
From your comments below it is now evident that the examples above need preparation since you are working with this Timepstamps. As a note, this is why giving enough information when asking the question is important (e.g. what kind of data you are dealing with). This is even moe important when it comes to dates as there are many formats. Here is an example with the date format you gave in the comments:
import datetime as dt
Date = '2020-09-03T16:18:38.929863799Z'
Date2 = '2020-10-03T16:18:38.929863799Z'
What you have here are Timestamps, so your first step is to convert them to datetime and then use to_pydate (It use to be called Timestamp.to_datetime() but is now deprecated.
Date = pd.to_datetime(Date)
Date2 = pd.to_datetime(Date2)
DATE_1 = Date.to_pydatetime()
DATE_2 = Date2.to_pydatetime()
after which you can compute the difference
DIFF = (pd.to_datetime(DATE_1) -pd.to_datetime(DATE_2))
which is Timedelta('-30 days +00:00:00')
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With