FutureWarning: frame.append method is deprecated, use pandas.concat instead [duplicate]

Question

This is my code:

def gettrigger(self):
    dfx = pd.DataFrame()
    for i in range(self.lags + 1):
        mask = (self.df['%K'].shift(i) < 20 ) & (self.df['%D'].shift(i) < 20)
        dfx = dfx.append(mask, ignore_index=True)
    return dfx.sum(axis=0)

After I execute this, i'm getting this warning:

The frame.append method is deprecated and will be removed from pandas
in a future version. Use pandas.concat instead.
  dfx = dfx.append(mask, ignore_index=True)

Can anyone help me?

Answer 1

so you have a loop:

dfx = pd.DataFrame()
for i in range(self.lags + 1):
    mask = (self.df["%K"].shift(i) < 20 ) & (self.df["%D"].shift(i) < 20)
    dfx = dfx.append(mask, ignore_index=True)

where you're accumulatively building a dataframe through append calls. This is inefficient! It has to build dataframes over and over, which is a costly process. And .append is getting deprecated anyway.

You should rather build a list of dataframes in the loop, and at the end concatanete them:

frames = []                    # <- switch to building a list
for i in range(self.lags + 1):
   mask = (self.df["%K"].shift(i) < 20 ) & (self.df["%D"].shift(i) < 20)
   frames.append(mask)        # <- save the new frames in that list

dfx = pd.concat(frames)       # <- at the end, concatanate them

(Note that the concatanation logic is after the loop; directly replacing .append calls inside the loop with pd.concat is inefficient too because re-building issue again.)

---

You can use a list comprehension or a generator expression to the same effect, depending on how complex your loop body is.

Above for example, can be written as:

dfx = pd.concat((self.df["%K"].shift(i) < 20 ) & (self.df["%D"].shift(i) < 20)
                for i in range(self.lags + 1))