Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Add minutes from another column to string time column in pyspark

I have the below pyspark dataframe.both are string columns.

time     additional_time_in_mins
11:00:00 60
13:00:00 60
14:00:00 30

I have to add the minutes in the additional time column to actual time and create an output as below in pyspark.

Expected output:

new_time
12:00:00
14:00:00
14:30:00

Is there a way to do this in pyspark

like image 979
Padfoot123 Avatar asked Dec 18 '25 19:12

Padfoot123


1 Answers

One simple option is converting time column to bigint in seconds using unix_timestamp function, add the minutes (minutes * 60s) and then cast the result back to timestamp.
Lastly, convert to hourly format.

df = df.withColumn('new_time', F.date_format((F.unix_timestamp('time', 'HH:mm:ss') + F.col('additional_time_in_mins')*60).cast('timestamp'), 'HH:mm:ss'))

df.show()

+--------+-----------------------+--------+
|    time|additional_time_in_mins|new_time|
+--------+-----------------------+--------+
|11:00:00|                     60|12:00:00|
|13:00:00|                     60|14:00:00|
|14:00:00|                     30|14:30:00|
+--------+-----------------------+--------+
like image 88
Ric S Avatar answered Dec 20 '25 10:12

Ric S



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!