Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to complement missing data in a time series using python?

I have a dataframe which has price for some days of the year, now I want to make a bigger datframe that shows all the days from the start of the year to some specific date. Then use the price for days that I already have in my original dataframe, and fill in between the days that have no price with the last price from that date.

as an example:

df = pd.DataFrame({
    'timestamps': pd.to_datetime(
        ['2021-01-04', '2021-01-07', '2021-01-14', '2021-01-21', '2021-01-28', '2021-01-29', 
'2021-02-04', '2021-02-12', '2021-02-18', '2021-02-25']),
    'LastPrice':['113.4377','115.0741','115.5709','116.5197','116.681','116.4198','117.5749','117.2175',
 '117.0541','117.5977']})

I want my new date series be like this

index=pd.date_range('2021-01-01', '2021-02-28')

dfObj = pd.DataFrame(columns=['new_Date','new_LastPrice'])
dfObj['new_Date'] = index

so, ideally I should have something like the following dataframe.(just the top part)

    new_Date    new_LastPrice
0   2021-01-01  0
1   2021-01-02  0
2   2021-01-03  0
3   2021-01-04  113.4377
4   2021-01-05  113.4377
5   2021-01-06  113.4377
6   2021-01-07  115.0741
7   2021-01-08  115.0741
8   2021-01-09  115.0741
9   2021-01-10  115.0741
10  2021-01-11  115.0741
11  2021-01-12  115.0741
12  2021-01-13  115.0741

Can anyone here help me with this, please?

like image 459
K saman Avatar asked Apr 07 '26 20:04

K saman


1 Answers

Use DataFrame.reindex with method='ffill':

index=pd.date_range('2021-01-01', '2021-02-28')

dfObj = (df.set_index('timestamps')
           .reindex(index, method='ffill')
           .fillna(0)
           .add_prefix('new_')
           .rename_axis('new_Date')
           .reset_index())
print (dfObj.head(13))
     new_Date new_LastPrice
0  2021-01-01             0
1  2021-01-02             0
2  2021-01-03             0
3  2021-01-04      113.4377
4  2021-01-05      113.4377
5  2021-01-06      113.4377
6  2021-01-07      115.0741
7  2021-01-08      115.0741
8  2021-01-09      115.0741
9  2021-01-10      115.0741
10 2021-01-11      115.0741
11 2021-01-12      115.0741
12 2021-01-13      115.0741
like image 105
jezrael Avatar answered Apr 09 '26 15:04

jezrael