Java's Spring Boot vs Python's FastAPI: Threads

Question

I'm a Java Spring boot developer and I develop 3-tier crud applications. I talked to a guy who seemed knowledgeable on the subject, but I didn't get his contact details. He was advocating for Python's FastAPI, because horizontally it scales better than Spring boot. One of the reasons he mentioned is that FastAPI is single-threaded. When the thread encounters a database lookup (or other work the can be done asyncly), it picks up other work to later return to the current work when the database results have come in. In Java, when you have many requests pending, the thread pool may get exhausted.

I don't understand this reasoning a 100%. Let me play the devil's advocate. When the Python program encounters an async call, it must somehow store the program pointer somewhere, to remember where it needs to continue later. I know that that place where the program pointer is stored is not at all a thread, but I have to give it some name, so let's call it a "logical thread". In Python , you can have many logical threads that are waiting. In Java, you can have a thread pool with many real threads that are waiting. To me, the only difference seems to be that Java's threads are managed at the operating system level, whereas Python's "logical threads" are managed by Python or FastAPI. Why are real threads that are waiting in a thread pool so much more expensive than logical threads that are waiting? If most of my threads are waiting, why can't I just increase the thread pool size to avoid exhaustion?

Answer 1

The issues with Java threads in the question are addressed by the project Loom, which is now included in Jdk21. It is very well explained here https://www.baeldung.com/openjdk-project-loom :

Presently, Java relies on OS implementations for both the continuation [of threads] and the scheduler [for threads].

Now, in order to suspend a continuation, it's required to store the entire call-stack. And similarly, retrieve the call-stack on resumption. Since the OS implementation of continuations includes the native call stack along with Java's call stack, it results in a heavy footprint.

A bigger problem, though, is the use of OS scheduler. Since the scheduler runs in kernel mode, there's no differentiation between threads. And it treats every CPU request in the same manner. (...) For example, consider an application thread which performs some action on the requests and then passes on the data to another thread for further processing. Here, it would be better to schedule both these threads on the same CPU. But since the [OS] scheduler is agnostic to the thread requesting the CPU, this is impossible to guarantee.

The question really boils down to Why are OS threads considered expensive?

Answer 2

FastAPI is a fast framework, and you can quickly (and easily) create API backends in it. To be honest, if you are a Java developer, I would recommend Quarkus or something for building a REST API, not FastAPI. FastAPI is a fantastic tool, absolutely great if you are already in the Python ecosystem.

When it goes about multithreading; Java is 'real' multithreading where as Python is very much not. Java threads will run concurrently; two tasks can and will be executed at the same time. In Python, within one Python process, this is (nearly) impossible. The reason for this is GIL (google it, there is ton's of stuff out there on how it works). The result is; even if you use 'real' threads in Python, code is still not executed concurrently but rather serially, where the interpreter (big difference to Java) is jumping from one call stack to another constantly.

As to what you refer to as 'logical threads', I think you mean the asynchronous capability of Python. This is basically the same as using threads (not really, but on an abstract level they are very similar); tasks are not run concurrently. There is just one thread that constantly switches between tasks. Tasks will yield back control to the event loop (the object that coordinates tasks and decides what is executed in which order), and another task is further executed until that task yields control, etc. It is basically the same kind of execution pattern as with threads within Python.

Comparing a Python framework to a Java framework is just weird in my opinion. They are both useful and cool, but not really competitors.