Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiprocessing in python vs number of cores

If a run a python script where i declare 6 processes using multiprocessing, but i only have 4 CPU cores, what happens to the additional 2 processes which can find a dedicated CPU core.

  1. How are they executed?
  2. If the two additional processes run as separate threads on the existing Cores, will GIL not stop their execution?

#Edit 1 - 21st Jan 2021

I have mixed up threads and processes in the question I asked. Since I have better clarity on the concept, I would rephrase question 2 as follows(for any future reference):

If the two additional processes run in parallel with two other processes in existing Cores, will GIL not stop their execution?

Ans: GIL does NOT affect the processes, GIL allows only one thread to run at a time, there is no restriction on processes, however. The system scheduler manages how the additional two processes will run on the existing cores.

like image 598
Samit Saxena Avatar asked Oct 20 '25 14:10

Samit Saxena


1 Answers

First you are mixing up threads and processes: in Python only threads not processes have to share a lock on their interpreter. If your are using the multiprocessing library then, your are using Python processes which have their own interpreter.

When you are using Python processes, their execution is managed by your operating system scheduler, in the same manner as every other processes in your computer. If you have more processes than CPU cores then the extra processes are waiting in background to be scheduled. This usually happen when an other process terminates, wait on an IO, or periodically with clock interrupts.

like image 82
lpeak Avatar answered Oct 22 '25 04:10

lpeak



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!