Display encoded video frames using React and Django

Question

I'm new to web development and have been trying to solve a problem for some time but no luck. I'm using React and Django

The thing is, there is a 3rd party application that performs some image processing using opencv on video frames and I've to display those encoded frames on the web browser.

I want to receive those frames using Django API, decode them, and display them using React JS, also returning a response with every frame to that 3rd party app.

I've prepared a flowchart of how things should work but haven't been able to start at all.

Flowchart:

The outcome on the browser should appear something like this.

Outcome:

Need to know how to approach this, shall I use WebSockets or can I send the encoded frames directly to React taking Django out of the picture.

Edit:

1. The frames will be served by the 3rd party app in cv2.imencode('.jpg', frame) encoded format along with some other data in a JSON packet.
2. The decoding needs to be done by Django or React (not sure which one will or should handle this)
3. The frames will keep on updating as if a real-time video is playing, ie. the moment a new frame is received, it must replace the old frame. The frame rate will be around 25 fps.
4. A response must be returned for each frame. Django needs to perform anything apart from serving frames and sending back a response.
5. Since there will be multiple cameras to be displayed simultaneously, do I need to use more than one port in Django or one is going to be sufficient?

Answer 1

You seem to be dealing with a video stream in MJPEG format (Motion JPEG). It is a sequence of JPEG frames without any inter-frame compression.

Frontend only

You typically can capture the MJPEG stream from the frontend. But if the third-party accesses the IP camera directly without caching layer, you might effectively DDOS it with very little traffic. I managed to slow down my localhost webcam MJPEG server by just using a handful of receivers.

Also, you directly expose your third-party source to your users. And the third party can monitor your users.

Backend-frontend

Passing the stream through your own backend is more costly on resources. You make minimum requests per frame to the third-party server but have to serve the stream to all your clients.

Resolution

If your frontend is going to be used by you only, go for the backend-free solution. If you have enough resources for the backend, you expect more clients, and you don't want to expose your clients to the third party, serve the MJPEG in the backend.

As for technical part, there are plenty out-of-the-box solutions.

• https://stackoverflow.com/a/34024692/20225232
• https://stackoverflow.com/a/34334188/20225232
• https://github.com/defvol/Paparazzo.js