Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to run curl commands parallel using GNU parallel

Just recently started programming in bash. I want to run curl command for 'n' times where n is user input. And I want to run in parallel. I came across GNU Parallel. the curl code for running n times is

ip1="some ip"
ip2="some ip"
ip3="some ip"
for ((j=1; j<= $servers;))
do
    for ((i= 1; i<= $val; i++))
    do
        c=ip$j
        value=scatest"$j$i"
        curl -s -X POST ${!c}:4000/invoke -H "content-type: application/json" -d '{"args":["org1","'$value'","27-06-2018"]}'
        if ((i % $val == 0))
        then
            ((j++))
        fi
    done
done

i want to run this curl command equally and parallel on these different hosts based on the user input. Can any one help me out?

Thanks in advance

like image 263
Lucky Avatar asked Jan 20 '26 23:01

Lucky


1 Answers

It is fairly simple to parallelize per server:

val=1000
doit() {
    server="$1"
    serverno="$2"
    for ((i= 1; i<= $val; i++))
    do
        value=scatest"$serverno$i"
        curl -s -X POST ${server}:4000/invoke -H "content-type: application/json" -d '{"args":["org1","'$value'","27-06-2018"]}'
    done
}
export -f doit
export val
parallel -j0 doit {} {#} ::: serverip1 serverip2 ...

If you can accept scatest has a different value you can parallelize even further:

doit() {
    server="$1"
    jobno="$2"
    i="$3"
    value=scatest"$jobno$i"
    curl -s -X POST ${server}:4000/invoke -H "content-type: application/json" -d '{"args":["org1","'$value'","27-06-2018"]}'
}
export -f doit
seq $val | parallel -j0 doit {1} {#} {2} ::: serverip1 serverip2 ... :::: -
like image 118
Ole Tange Avatar answered Jan 23 '26 13:01

Ole Tange



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!