is there a way to partially return results from Python web service....?

Question

I am new to python. I am using Flask for creating a web service which makes lots of api calls to linkedin. The problem with this is getting the final result set lot of time and frontend remains idle for this time. I was thinking of returning partial results found till that point and continuing api calling at server side. Is there any way to do it in Python? Thanks.

Answer 1

Flask has the ability to stream data back to the client. Sometimes this requires javascript modifications to do what you want but it is possible to send content to a user in chunks using flask and jinja2. It requires some wrangling but it's doable.

A view that uses a generator to break up content could look like this (though the linked to SO answer is much more comprehensive).

from flask import Response

@app.route('/image')
def generate_large_image():
    def generate():
        while True:
            if not processing_finished():
                yield ""
            else:
                yield get_image()
    return Response(generate(), mimetype='image/jpeg')

Answer 2

There are a few ways to do this. The simplest would be to return the initial request via flask immediately and then use Javascript on the page you returned to make an additional request to another URL and load that when it comes back. Maybe displaying a loading indicator or something.

The additional URL would look like this

@app.route("/linkedin-data")
def linkedin():
    # make some call to the linked in api which returns "data", probably in json
    return flask.jsonify(**data)

Fundamentally, no. You can't return a partial request. So you have to break your requests up into smaller units. You can stream data using websockets. But you would still be sending back an initial request, which would then create a websocket connection using Javascript, which would then start streaming data back to the user.