Is there a way to decrease the frequency with which curl updates its progress meter?
I need to download a large file in a CI pipeline. I want to see progress updates periodically, so that the CI pipeline is not automatically marked as "stuck". However, by default, curl updates its status quite often, so I end up with a pipeline log with thousands of lines like the following:
[...]
33 189G 33 64.0G 0 0 8107k 0 6:48:27 2:18:09 4:30:18 4580k
33 189G 33 64.1G 0 0 8107k 0 6:48:28 2:18:10 4:30:18 4901k
33 189G 33 64.1G 0 0 8107k 0 6:48:29 2:18:11 4:30:18 5312k
33 189G 33 64.1G 0 0 8106k 0 6:48:31 2:18:12 4:30:19 5003k
33 189G 33 64.1G 0 0 8106k 0 6:48:32 2:18:13 4:30:19 4991k
33 189G 33 64.1G 0 0 8105k 0 6:48:34 2:18:14 4:30:20 4286k
33 189G 33 64.1G 0 0 8104k 0 6:48:36 2:18:15 4:30:21 3788k
33 189G 33 64.1G 0 0 8104k 0 6:48:38 2:18:16 4:30:22 3057k
33 189G 33 64.1G 0 0 8103k 0 6:48:40 2:18:17 4:30:23 2910k
33 189G 33 64.1G 0 0 8103k 0 6:48:41 2:18:18 4:30:23 2916k
33 189G 33 64.1G 0 0 8102k 0 6:48:43 2:18:19 4:30:24 3460k
33 189G 33 64.1G 0 0 8102k 0 6:48:44 2:18:20 4:30:24 3621k
33 189G 33 64.1G 0 0 8101k 0 6:48:45 2:18:21 4:30:24 3953k
33 189G 33 64.1G 0 0 8101k 0 6:48:47 2:18:22 4:30:25 4393k
33 189G 33 64.1G 0 0 8100k 0 6:48:48 2:18:23 4:30:25 4392k
33 189G 33 64.1G 0 0 8100k 0 6:48:50 2:18:24 4:30:26 4392k
33 189G 33 64.1G 0 0 8099k 0 6:48:51 2:18:25 4:30:26 4394k
[...]
No there isn't any such option to curl.
curl will update the progress meter at least once every second but when there's data being transferred it might do it (much) more frequently.
I've added the idea in the curl TODO document!
The best I could come up with is to silence curl altogether, and instead start a separate process which reports the file size every minute or longer.
(while true ; do sleep 60; ls -lSh filename.ext ; done) &
monitor_pid=$!
curl -O -C - --retry 999 --retry-max-time 0 --no-progress-meter ...
kill $monitor_pid
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With