Integrate Jupyter notebook with data fetched from Django application

Question

I have a Django application that integrates data from a SQL database and displays that as a downloadable HTML table. Now I would also like to add analysis functionality, but instead of painstakingly adding only a few functionalities using JavaScript, I want to redirect the user to a Jupyter notebook, so the user has full access to the functionality of Python data analysis libraries or even other languages. Now I'm not sure how to approach this and I have several questions:

• Am I right that Jupyter needs to be run on a different server than the Django app since it uses the Tornado server?
• How would I transfer the data produced with Django to Jupyter? I store the data in a new MySQL table, but I would at least need to transfer the table name.
• Given the table name was transferred to Jupyter I would still need to execute custom code to access the database. I found in this question that some defaults for code execution can be set in IPython configuration, but since the table from which the data should be loaded is never the same, this would have to be adapted dynamically.

I'm glad for any suggestions and comments on whether this idea makes sense at all.

Answer 1

Am I right that Jupyter needs to be run on a different server than the Django app since it uses the Tornado server?

No. They can run on the same server.

How would I transfer the data produced with Django to Jupyter? I store the data in a new MySQL table, but I would at least need to transfer the table name.

You can run django and Jupyter together and execute any django code from Jupyter. The easiest way to do so is by using shell_plus from the package django-extensions You can even connect to the production database.

You should consider the possible security and data safety risks. This is a potential vector for remote code execution and data leaks.

It's probably safer to spin up a cloned django instance with a cloned database or use a different django configuration (settings.py) with read-only access to the db.

It's quite easy to mess up production data from Jupyter, since you can do stuff such as User.objects.all().delete() And that would be a problem...

Also consider running this as a user with stricter read and write permissions than you use for your regular django app.

And of course, you should make sure that the Jupyter site is not exposed on a publicly accessible url.