Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Decrease the frequency of curl progress updates

Tags:

curl

Is there a way to decrease the frequency with which curl updates its progress meter?

I need to download a large file in a CI pipeline. I want to see progress updates periodically, so that the CI pipeline is not automatically marked as "stuck". However, by default, curl updates its status quite often, so I end up with a pipeline log with thousands of lines like the following:

[...]
 33  189G   33 64.0G    0     0  8107k      0  6:48:27  2:18:09  4:30:18 4580k
 33  189G   33 64.1G    0     0  8107k      0  6:48:28  2:18:10  4:30:18 4901k
 33  189G   33 64.1G    0     0  8107k      0  6:48:29  2:18:11  4:30:18 5312k
 33  189G   33 64.1G    0     0  8106k      0  6:48:31  2:18:12  4:30:19 5003k
 33  189G   33 64.1G    0     0  8106k      0  6:48:32  2:18:13  4:30:19 4991k
 33  189G   33 64.1G    0     0  8105k      0  6:48:34  2:18:14  4:30:20 4286k
 33  189G   33 64.1G    0     0  8104k      0  6:48:36  2:18:15  4:30:21 3788k
 33  189G   33 64.1G    0     0  8104k      0  6:48:38  2:18:16  4:30:22 3057k
 33  189G   33 64.1G    0     0  8103k      0  6:48:40  2:18:17  4:30:23 2910k
 33  189G   33 64.1G    0     0  8103k      0  6:48:41  2:18:18  4:30:23 2916k
 33  189G   33 64.1G    0     0  8102k      0  6:48:43  2:18:19  4:30:24 3460k
 33  189G   33 64.1G    0     0  8102k      0  6:48:44  2:18:20  4:30:24 3621k
 33  189G   33 64.1G    0     0  8101k      0  6:48:45  2:18:21  4:30:24 3953k
 33  189G   33 64.1G    0     0  8101k      0  6:48:47  2:18:22  4:30:25 4393k
 33  189G   33 64.1G    0     0  8100k      0  6:48:48  2:18:23  4:30:25 4392k
 33  189G   33 64.1G    0     0  8100k      0  6:48:50  2:18:24  4:30:26 4392k
 33  189G   33 64.1G    0     0  8099k      0  6:48:51  2:18:25  4:30:26 4394k
[...]
like image 646
ostrokach Avatar asked Feb 28 '26 07:02

ostrokach


2 Answers

No there isn't any such option to curl.

curl will update the progress meter at least once every second but when there's data being transferred it might do it (much) more frequently.

I've added the idea in the curl TODO document!

like image 161
Daniel Stenberg Avatar answered Mar 02 '26 23:03

Daniel Stenberg


The best I could come up with is to silence curl altogether, and instead start a separate process which reports the file size every minute or longer.

(while true ; do sleep 60; ls -lSh filename.ext ; done) &
monitor_pid=$!
curl -O -C - --retry 999 --retry-max-time 0 --no-progress-meter ...
kill $monitor_pid
like image 38
ostrokach Avatar answered Mar 02 '26 23:03

ostrokach



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!