Python: How many cores are used by my python program with five processes?

Question

I have a python program consisting of 5 processes outside of the main process. Now I'm looking to get an AWS server or something similar on which I can run the script. But how can I find out how many vCPU cores are used by the script/how many are needed? I have looked at:

import multiprocessing

multiprocessing.cpu_count()

But it seems that it just returns the CPU count that's on the system. I just need to know how many vCPU cores the script uses.

Thanks for your time.

EDIT:

Just for some more information. The Processes are running indefinitely.

Answer 1

On Linux you can use the "top" command at the command line to monitor the real-time activity of all threads of a process id:

top -H -p <process id>

Answer 2

Answer to this post probably lies in the following question:

Multiprocessing : More processes than cpu.count

In short, you have probably hundreds of processes running, but that doesn't mean you will use hundreds of cores. It all depends on utilization, and the workload of the processes.

You can also get some additional info from the psutil module

import psutil

print(psutil.cpu_percent())
print(psutil.cpu_stats())
print(psutil.cpu_freq())

or using OS to receive current cpu usage in python:

import os
import psutil

l1, l2, l3 = psutil.getloadavg()
CPU_use = (l3/os.cpu_count()) * 100

print(CPU_use)

• Credit: DelftStack

Edit

There might be some information for you in the following medium article. Maybe there are some tools for CPU usage too. https://medium.com/survata-engineering-blog/monitoring-memory-usage-of-a-running-python-program-49f027e3d1ba

Edit 2

A good guideline for how many processes to start depends on the amount of threads available. It's basically just Thread_Count + 1, this ensures your processor doesn't just 'sit around and wait', this however is best used when you are IO bound, think of waiting for files from disk. Once it waits, that process is locked, thus you have 8 others to take over. The one extra is redundancy, in case all 8 are locked, the one that's left can take over right away. You can however in- or decrease this if you see fit.