Python-MultiThreading: Can MultiThreading improve "for loop" performance?

Question

As far as my understanding:

• MultiThread is an ideal option for I/O applications.

Therefore, I test a "for loop" code without any I/O. (As following code)

Howerver, it can reduce the execution time from 6.3s to 3.7s.

Is the result correct ?

or any mistake in my suppose ?

from multiprocessing.dummy import Pool as ThreadPool
import time

# normal
l = []
s = time.time()
for i in range(0, 10000):
    for j in range(i):
        l.append(j * 10)

e = time.time()
print(f"case1: {e-s}") # 6.3 sec

# multiThread
def func(x):
    for i in range(x):
        l_.append(i * 10)

with ThreadPool(50) as pool:
    l_ = []
    s = time.time()

    pool.map(func, range(0, 10000))

    e = time.time()
    print(f"case2: {e-s}") # 3.7 sec

Answer 1

what you are seeing is just python specializing the function by using faster op-codes for the multithreaded version as it is a function that is called multiple times, See PEP 659 Specializing Adaptive Interpreter, this only true for python 3.11 and later.

changing the non-multithreaded version to also be a function that is called multiple times give almost the same performance (python 3.11)

from multiprocessing.dummy import Pool as ThreadPool
import time

l = []
def f2(i):
    for j in range(i):
        l.append(j * 10)
def f1():
    # normal
    for i in range(0, 10_000):
        f2(i)

s = time.time()
f1()
e = time.time()
print(f"case1: {e-s}")


# multiThread
def func(x):
    global l_
    for i in range(x):
        l_.append(i * 10)

with ThreadPool(50) as pool:
    l_ = []
    s = time.time()

    pool.map(func, range(0, 10_000))

    e = time.time()
    print(f"case2: {e-s}")

case1: 3.9744303226470947
case2: 4.036579370498657

threading in python IS going to be slower for functions that need the GIL, and manipulating lists requires the GIL so using threading will be slower, python threads only improve performance if the GIL is dropped (which happens on IO and C external libraries calls), if this is ever not the case then either your code drops the GIL or your benchmark is flawed.