while looking on how to parallelize bash tasks, I've stumbles over a code like this:
for item in "${items[@]}"
do
((i=i%THREADS)); ((i++==0)) && wait
process_item $item &
done
Where process_item is some king of function/program that works with item and the THREADS var contain the maximum number of background processes that can run simultaneously.
Can someone explain to me how this works? I understand that i=i%THREADS ensures that i is between 0 and THREADS-1, and that i++==0 increments i and checks whether it is 0. But is wait bound to all sub processes? Or how does it know that is has to wait until the previous batch stopped processing?
It's an obfuscated way of writing
for item in "${items[@]}"
do
# Every THREADSth job, stop and wait for everything
# to complete.
if (( i % THREADS == 0 )); then
wait
fi
((i++))
process_item $item &
done
It also doesn't actually work terribly well. It doesn't ensure that there are always $THREADS jobs running, only that no more than $THREADS jobs are running at once.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With