Monitoring progress within xargs

Question

Is there any way to track or monitor progress within xargs?

I'm replacing a for loop with find … | xargs for performance (specifically to run tasks in parallel). I've read that parallel has a progress flag, but I was wondering if there was a way to accomplish this strictly using xargs.

I'm also aware that tasks run in parallel via xargs will not necessarily finish in the correct order, which compounds the complexity of monitoring progress. Even if a solution would give me a general idea of the progress, that would be a great start.

Answer 1

If you just want to input how many lines you roughly already processed you may create simple shell function to do that

#!/bin/bash

#-----
##
## @function count
##
## @desc Write every n'th number (if n is 5 write 5, 10, 15, ...)
##
## @param $1 - number 
##
#-----
function count {
   typeset C=0
   while read L; do
      C=$(( C + 1 ))
      if [ $(( $C % $1 )) -eq 0 ]; then
         echo $C 1>&2
      fi
      echo "$L"
   done
}

find . | count 100 | xargs ...

Small problem is that this prints number of lines passed to xargs, not number of lines already processed by the command invoked by xargs. Plus every pipe has some buffer, so it will show slightly higher number than it should. On my machine it showed ~500 lines in advance to real state but if you are processing sufficiently large number of lines 500 is negligible :)