Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How limiting background jobs works?

while looking on how to parallelize bash tasks, I've stumbles over a code like this:

for item in "${items[@]}"
do
  ((i=i%THREADS)); ((i++==0)) && wait
  process_item $item &
done

Where process_item is some king of function/program that works with item and the THREADS var contain the maximum number of background processes that can run simultaneously.

Can someone explain to me how this works? I understand that i=i%THREADS ensures that i is between 0 and THREADS-1, and that i++==0 increments i and checks whether it is 0. But is wait bound to all sub processes? Or how does it know that is has to wait until the previous batch stopped processing?

like image 596
Uko Avatar asked Dec 20 '25 13:12

Uko


1 Answers

It's an obfuscated way of writing

for item in "${items[@]}"
do
    # Every THREADSth job, stop and wait for everything
    # to complete.
    if (( i % THREADS == 0 )); then
        wait
    fi
    ((i++))
    process_item $item &
done

It also doesn't actually work terribly well. It doesn't ensure that there are always $THREADS jobs running, only that no more than $THREADS jobs are running at once.

like image 178
chepner Avatar answered Dec 23 '25 05:12

chepner



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!