Iterate over files in directory using bash script

Question

I would like to iterate over the files in given directory. I tried using the for loop for the same... but I have one another loop inside this loop where I need to read multiple files till the condition is true within that loop to upload at a time... but withing inner loop I am able to access only one file since file iterator loop is outside the inner loop. Is there any thing like file.getNext so that i can access number of files inside the loop without using index/loop.

I need something like this:

while [ count -le $uploadRate ]
do
   files="$files,directory.getNextFile"
   $(curl -o $log_Path -T "{$files}" url)
done

where $files will have number of files uploaded at a time I don't want to load the files into array since directory has huge number of files, so array is not feasible option, so would like to iterate over directory like getNext file.

Answer 1

You can keep your outer loop like this for reading all the files/directories in a directory /somedir/:

filesToBeRead=5
chunkFiles=()
for f in /somedir/*; do
    chunkFiles+=( "$f" )
    if [[ ${#chunkFiles[@]} -eq 5 ]]; then
        processChunkFiles( "${chunkFiles[@]}" )
        chunkFiles=()
    fi
done
processChunkFiles( "${chunkFiles[@]}" )

PS: You need to write a processChunkFiles function to handle/process 5 files.

Answer 2

Put the filenames in an array, and iterate through it in groups of 5:

files=(/path/to/dir/*)
nfiles=${#files[@]}
for ((i = 0; i < nfiles; i += 5)); do
    for ((j = i; j < i+5; j++)); do
        # Do something with ${files[$j]}
    done
done