Parallel computing in Julia - running a simple for-loop on multiple cores

Question

For starters, I have to say I'm completely new to parallel computing (and know close to nothing about computer science), so my understanding of what things like "workers" or "processes" actually are is very limited. I do however have a question about running a simple for-loop that presumably has no dependencies between the iterations in parallel.

Let's say I wanted to do the following:

for N in 1:5:20
    println("The N of this iteration in $N")
end

If I simply wanted these messages to appear on screen and the order of appearance didn't matter, how could one achieve this in Julia 0.6, and for future reference in Julia 0.7 (and therefore 1.0)?

Answer 1

Distributed Processing

Start julia with e.g. julia -p 4 if you want to use 4 cpus (or use the function addprocs(4)). In Julia 1.x, you make a parallel loop as following:

using Distributed
@distributed for N in 1:5:20
    println("The N of this iteration in $N")
end

Note that every process have its own variables per default. For any serious work, have a look at the manual https://docs.julialang.org/en/v1/manual/parallel-computing/, in particular the section about SharedArrays.

Another option for distributed computing are the function pmap or the package MPI.jl.

Threads

Since Julia 1.3, you can also use Threads as noted by wueli. Start julia with e.g. julia -t 4 to use 4 threads. Alternatively you can or set the environment variable JULIA_NUM_THREADS before starting julia. For example Linux/Mac OS:

export JULIA_NUM_THREADS=4 

In windows, you can use set JULIA_NUM_THREADS 4 in the cmd prompt.

Then in julia:

   Threads.@threads for N = 1::20
       println("N = $N (thread $(Threads.threadid()) of out $(Threads.nthreads()))")
   end

All CPUs are assumed to have access to shared memory in the examples above (e.g. "OpenMP style" parallelism) which is the common case for multi-core CPUs.

Answer 2

Just to add the example to the answer of Chris. Since the release of julia 1.3 you do this easily with Threads.@threads

Threads.@threads for N in 1:5:20
    println("The number of this iteration is $N")
end

Here you are running only one julia session with multiple threads instead of using Distributed where you run multiple julia sessions.

See, e.g. multithreading blog post for more information.