How to give status during a post request in flask?

Question

I've created a page that allows a user to upload an excel file, which is then parsed into it's columns and then the rows are inserted into the database 500 rows at a time.

This is isn't a terribly long process - between 25 and 90 seconds, but long enough that I would like to give the user some feedback to let them know it's actually still working - in the form of status messages and/or progress bars.

My app is written in flask like this:

app.py

from flask import Flask, render_template, request
import tqdm 

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def fun():
    if request.method == 'GET':
        return render_template('index.html')
    else:
        filename = request.form['filename']
        print('opening file') #change from console print to webpage
        df = pandas.read_excel(filename)
        print('File read.. processing data\n') #change from console to webpage
        processData()
        print('connecting to db....\n') #change to webpage print
        db.connect()
        print('connected to db! inserting rows') #change to webpage print
        bulk_inserts = rows/500
        for i in tqdm(range(bulk_inserts)): #wrapping tqdm around range makes a progress bar
            insert500rows() 
        db.commit()
        db.disconnect()
        return 'Complete. ' + str(rows) + ' inserted.' #this gets sent as a post to the page

app.run()

I know you can only send one response to a post request, but how can I give the user status of the process if I can only send one response? Maybe I'm going about this the wrong way, but I think this is a pretty common use case. How else should I set this up if this way won't work?

---

For some reason marked as a duplicate of this question. That question asks how to print a continuous stream of values to a screen. Here I am asking how to send a message at certain points of execution. I think the comments provided about Flask-socketio provide a different approach to a different problem.

Answer 1

The "one response to one request" is a matter of how HTTP protocol works: the client sends a query and some data (the POST request), and the server responds with some other data (your "one response"). While you could technically get the server to send back pieces of the response in chunks, that is not how it works in practice; for one, browsers don't handle that too well.

You need to do this a different way. For instance, create a "side channel" with SocketIO, as the commenters helpfully suggest. Then you can send updates to the client through this side channel - instead of your prints, you would use socketio.emit.

On the client side, you would first subscribe to a SocketIO channel when the page loads. Then you would submit the file through an AJAX call (or in an separate iframe), and keep the SocketIO connection open on the page, to display the updates.

This way the POST request is separated from your "page loading". The javascript on the page remains active and can read and display progress update, and the upload (with the associated wait time) happens in background.