How to achieve Async file download with TPL in C# [closed]

Question

I have a windows console client that sleeps for few minutes and on waking up it requests new updates from Rest API. The request generates a response with json data involves a list of objects, each object consists of ID, description, screenshot being sent from the API to the client as URL. the console application is required to consume the json response, and for each object, it looks up the URL and try to download the corresponding image associated with each object in the list. The code is expressed as follows

foreach (var jobject in response)
{
    Console.WriteLine(jobject.id);
    Console.WriteLine(jobject.description);
    if (jobject.shotUrl != null)
    {
        WebClient webclient = new WebClient();
        webclient.DownloadFileAsync(new System.Uri(jobject.shotUrl), "F:\\" + jobject.id + ".jpg");
    }
}

Sometimes there can be around 500 json objects which means 500 photo download ... again means creating 500 webclients. I feel this is not good idea.

My question is: can I gain performance if I rely on TPL? how to do so ?

Answer 1

Using HttpClient is much easier for concurrency, you can declare one private HttpClient for the class:

System.Net.Http.HttpClient _client = new System.Net.Http.HttpClient();

And then you can write a method that downloads the files and saves them to disk like so:

 private async Task DownloadFile(string shortUrl, string destination)
 {
     using (var response = await _client.GetStreamAsync(shortUrl))
     using (var fileStream = File.Create(destination))
     {
         await response.CopyToAsync(fileStream);
         await fileStream.FlushAsync();
     }
 }

Then you can use it like this:

try
{
     await DownloadFile(jobject.shortUrl, "F:\\" + jobject.id + ".jpg");
}
catch (Exception e)
{
     // Do appropriate exception handling
}

And if you want to download all the files in parallel, You can use Task.WhenAll()

try
{
     var tasks = response.Select(j => DownloadFile(j.shortUrl, "F:\\" + j.id + ".jpg"));
     await Task.WhenAll(tasks);
}
catch (Exception e)
{
     // Do appropriate exception handling
}