Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas join two dataframe based on Index

I am trying to join two dataframes based on Index values of dataframes however, my result produces NaN and followed by values with index values repeated.

print st

results in (stored in data frame series and with only index as follows)

index
1499054400000
1499140800000
1499227200000
1499313600000
1499400000000

Another dataframe is as follows

print dtest2

This results in (same index as above)

                1
index   
1499227200000   33.48
1499400000000   35.71

I am trying to get the following when i merge two dataframes

Result desired

index            1
1499054400000    0.0
1499140800000    0.0
1499227200000    33.48
1499313600000    0.0
1499400000000    35.71

However when i concatenate as follows

pd.concat([st,dtest2],ignore_index=False)

I get the following

                1
index   
1499054400000   NaN
1499140800000   NaN
1499227200000   NaN
1499313600000   NaN
1499400000000   NaN
1499227200000   33.48
1499400000000   35.71
like image 714
abhi_phoenix Avatar asked Sep 06 '25 00:09

abhi_phoenix


1 Answers

IIUC... use pd.DataFrame.reindex

dtest2.reindex(st.index, fill_value=0)

                   1
index               
1499054400000   0.00
1499140800000   0.00
1499227200000  33.48
1499313600000   0.00
1499400000000  35.71

However, you may have a hidden problem in that the types are not the same. You can ensure that all indices are the same by using astype to cast them.

dtest2.index = dtest2.index.astype(str)
st.index = st.index.astype(str)

dtest2.reindex(st.index, fill_value=0)

                   1
index               
1499054400000   0.00
1499140800000   0.00
1499227200000  33.48
1499313600000   0.00
1499400000000  35.71

Or

dtest2.index = dtest2.index.astype(int)
st.index = st.index.astype(int)

dtest2.reindex(st.index, fill_value=0)

                   1
index               
1499054400000   0.00
1499140800000   0.00
1499227200000  33.48
1499313600000   0.00
1499400000000  35.71
like image 107
piRSquared Avatar answered Sep 08 '25 12:09

piRSquared