使用HttpClient的异步文件下载 [英] Using HttpClient for Asynchronous File Downloads

问题描述

我有它返回一个CSV文件POST请求的服务。我想使用异步技术来下载文件说。虽然我可以得到的文件，我的code有几个突出的问题和疑问：

1）这是真的异步？

2）是否有办法知道内容的长度，即使它在分块格式发送？想想进度条）。

3）如何可以为了抵挡程序退出，直到所有工作完成最好的我监测进展情况。

 使用系统;
使用System.IO;
使用System.Net.Http;命名空间TestHttpClient2
{
    类节目
    {
        / *
         *使用雅虎门户网站访问报价的股票 - 执行异步操作。
         * /        静态字符串的baseUrl =htt​​p://real-chart.finance.yahoo.com/;
        静态字符串requestUrlFormat =/table.csv?s={0}&d=0&e=9&f=2015&g=d&a=4&b=5&c=2000&ignore=.csv ;        静态无效的主要（字串[] args）
        {
            而（真）
            {
                Console.Write（输入一个符号来研究或[Enter]键退出：）;
                串符号=到Console.ReadLine（）;
                如果（string.IsNullOrEmpty（符号））
                    打破;
                DownloadDataForStockAsync（符号）;
            }
        }        静态异步无效DownloadDataForStockAsync（串符号）
        {
            尝试
            {
                使用（VAR的客户=新的HttpClient（））
                {
                    client.BaseAddress =新的URI（的baseUrl）;
                    client.Timeout = TimeSpan.FromMinutes（5）;
                    字符串requestUrl =的String.Format（requestUrlFormat，符号）;                    // VAR内容=新KeyValuePair＆LT;字符串，字符串＆GT; [] {
                    //};
                    //变种formUrlEn codedContent =新FormUrlEn codedContent（内容）;                    VAR要求=新的Htt prequestMessage（HttpMethod.Post，requestUrl）;
                    VAR sendTask = client.SendAsync（请求，HttpCompletionOption.ResponseHeadersRead）;
                    VAR响应= sendTask.Result.EnsureSuccessStatus code（）;
                    VAR httpStream =等待response.Content.ReadAsStreamAsync（）;                    字符串输出目录=StockQuotes;                    如果（！Directory.Exists（输出目录））
                    {
                        Directory.CreateDirectory（输出目录）;
                    }                    日期时间currentDateTime = DateTime.Now;
                    VAR文件路径= Path.Combine（输出目录，的String.Format（{1：D4} _ {2} D2 _ {3} D2 _ {4} D2 _ {5} D2 _ {6：D2} _ {7：D3} _ {0}的.csv
                        符号，
                        currentDateTime.Year，currentDateTime.Month，currentDateTime.Day，
                        currentDateTime.Hour，currentDateTime.Minute，currentDateTime.Second，currentDateTime.Millisecond
                        ））;                    使用（VAR FILESTREAM = File.Create（文件路径））
                    使用（VAR读者=新的StreamReader（httpStream））
                    {
                        httpStream.CopyTo（FILESTREAM）;
                        fileStream.Flush（）;
                    }
                }
            }
            赶上（异常前）
            {
                Console.WriteLine（错误，请重试！）;
            }
        }    }
}
 

解决方案

  

  
1. 这真的是异步的？

  

是的，大部分。在 DownloadDataForStockAsync（）方法将在手术前恢复完成后，在等待response.Content.ReadAsStreamAsync（）语句

主要的例外是附近的方法的末尾，你叫 Stream.CopyTo（）。这是不同步的，并且因为它可能会导致明显的延迟一个潜在的冗长的操作。然而，在一个控制台程序，你不会注意到，因为该方法的延续线程池中，而不是原来的调用线程执行。

如果您打算在此code移动到一个GUI框架，如的WinForms或WPF，你应该改变语句来读等待httpStream.CopyToAsync（FILESTREAM）;

  <醇开始=2>
  

有没有办法知道内容的长度，即使它在分块格式发送？想想进度条）。

  

假设服务器包括报头中的的Content-Length （它应该），是的。这应该是可能的。

请注意，如果你使用的HttpWebRequest ，响应对象将有一个 CONTENTLENGTH 属性直接给你这个值。您正在使用的Htt prequestMessage ，而不是在这里，这我不熟悉。但是尽可能接近我可以告诉，你应该能够访问这样的的Content-Length 值：

 长？ CONTENTLENGTH = response.Content.Headers.ContentLength;如果（CONTENTLENGTH！= NULL）
{
    //使用价值进行初始化确定的进度指示
}
其他
{
    //没有内容长度提供的;需要将显示为不确定的进展
}
 

  <醇开始=3>
  

如何才能抵挡程序退出，直到所有工作完成最好的监测进展情况。

  

有很多方法。我会指出，任何合理的方式将要求您更改 DownloadDataForStockAsync（）方法，以便返回工作和没有无效。否则，你就无法获得所创建的任务。你应该做的，虽然这无论如何，所以这不是什么大不了的事。 ：）

最简单的将只是保持你开始的任务列表，然后退出之前对他们等待：

 静态无效的主要（字串[] args）
{
    清单＆LT;任务＆GT;任务=新的List＆LT;任务＆GT;（）;    而（真）
    {
        Console.Write（输入一个符号来研究或[Enter]键退出：）;
        串符号=到Console.ReadLine（）;
        如果（string.IsNullOrEmpty（符号））
            打破;
        tasks.Add（DownloadDataForStockAsync（符号））;
    }    Task.WaitAll（任务）;
}
 

当然，这需要你明确地维护每个工作对象的列表，包括那些已经完成。如果打算为这个以很长一段时间运行和处理非常大的数目的符号，这可能是令人望而却步。在这种情况下，你可能会preFER使用 CountDownEvent 对象：

 静态无效的主要（字串[] args）
{
    CountDownEvent countDown方法=新CountDownEvent（）;    而（真）
    {
        Console.Write（输入一个符号来研究或[Enter]键退出：）;
        串符号=到Console.ReadLine（）;
        如果（string.IsNullOrEmpty（符号））
            打破;        countDown.AddCount（）;
        DownloadDataForStockAsync（符号）.ContinueWith（任务= GT; countdown.Signal（））;
    }    countDown.Wait（）;
}
 

这只是增加你创建的每个任务的 CountDownEvent 柜台，并附加延续到每个任务递减计数器。当计数器达到零，事件设置，允许）的调用等待（返回。

I have a service which returns a csv file to a POST request. I would like to download said file using asynchronous techniques. While I can get the file, my code has a couple of outstanding problems and questions:

1) Is this really asynchronous?

2) Is there a way to know the length of the content even though it is being sent in chunked format? Think progress bars).

3) How can I best monitor progress in order to hold off the program exit until all work is complete.

using System;
using System.IO;
using System.Net.Http;

namespace TestHttpClient2
{
    class Program
    {
        /*
         * Use Yahoo portal to access quotes for stocks - perform asynchronous operations.
         */

        static string baseUrl = "http://real-chart.finance.yahoo.com/";
        static string requestUrlFormat = "/table.csv?s={0}&d=0&e=9&f=2015&g=d&a=4&b=5&c=2000&ignore=.csv";

        static void Main(string[] args)
        {
            while (true) 
            {
                Console.Write("Enter a symbol to research or [ENTER] to exit: ");
                string symbol = Console.ReadLine();
                if (string.IsNullOrEmpty(symbol))
                    break;
                DownloadDataForStockAsync(symbol);
            }
        }

        static async void DownloadDataForStockAsync(string symbol)
        {
            try
            {
                using (var client = new HttpClient())
                {
                    client.BaseAddress = new Uri(baseUrl);
                    client.Timeout = TimeSpan.FromMinutes(5);
                    string requestUrl = string.Format(requestUrlFormat, symbol);

                    //var content = new KeyValuePair<string, string>[] {
                    //    };
                    //var formUrlEncodedContent = new FormUrlEncodedContent(content);

                    var request = new HttpRequestMessage(HttpMethod.Post, requestUrl);
                    var sendTask = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
                    var response = sendTask.Result.EnsureSuccessStatusCode();
                    var httpStream = await response.Content.ReadAsStreamAsync();

                    string OutputDirectory = "StockQuotes";

                    if (!Directory.Exists(OutputDirectory))
                    {
                        Directory.CreateDirectory(OutputDirectory);
                    }

                    DateTime currentDateTime = DateTime.Now;
                    var filePath = Path.Combine(OutputDirectory, string.Format("{1:D4}_{2:D2}_{3:D2}_{4:D2}_{5:D2}_{6:D2}_{7:D3}_{0}.csv",
                        symbol,
                        currentDateTime.Year, currentDateTime.Month, currentDateTime.Day,
                        currentDateTime.Hour, currentDateTime.Minute, currentDateTime.Second, currentDateTime.Millisecond
                        ));

                    using (var fileStream = File.Create(filePath))
                    using (var reader = new StreamReader(httpStream))
                    {
                        httpStream.CopyTo(fileStream);
                        fileStream.Flush();
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error, try again!");
            }
        }

    }
}

解决方案

1. "Is this really asynchronous?"

Yes, mostly. The DownloadDataForStockAsync() method will return before the operation is complete, at the await response.Content.ReadAsStreamAsync() statement.

The main exception is near the end of the method, where you call Stream.CopyTo(). This isn't asynchronous, and because it's a potentially lengthy operation could result in noticeable delays. However, in a console program you won't notice, because the continuation of the method is executed in the thread pool rather than the original calling thread.

If you intend to move this code to a GUI framework, such as Winforms or WPF, you should change the statement to read await httpStream.CopyToAsync(fileStream);

2. Is there a way to know the length of the content even though it is being sent in chunked format? Think progress bars).

Assuming the server includes the Content-Length in the headers (and it should), yes. This should be possible.

Note that if you were using HttpWebRequest, the response object would have a ContentLength property giving you this value directly. You are using HttpRequestMessage here instead, which I'm less familiar with. But as near as I can tell, you should be able to access the Content-Length value like this:

long? contentLength = response.Content.Headers.ContentLength;

if (contentLength != null)
{
    // use value to initialize "determinate" progress indication
}
else
{
    // no content-length provided; will need to display progress as "indeterminate"
}

3. How can I best monitor progress in order to hold off the program exit until all work is complete.

There are lots of ways. I will point out that any reasonable way will require that you change the DownloadDataForStockAsync() method so that it returns Task and not void. Otherwise, you don't have access to the task that's created. You should do this anyway though, so that's not a big deal. :)

The simplest would be to just keep a list of all the tasks you start, and then wait on them before exiting:

static void Main(string[] args)
{
    List<Task> tasks = new List<Task>();

    while (true) 
    {
        Console.Write("Enter a symbol to research or [ENTER] to exit: ");
        string symbol = Console.ReadLine();
        if (string.IsNullOrEmpty(symbol))
            break;
        tasks.Add(DownloadDataForStockAsync(symbol));
    }

    Task.WaitAll(tasks);
}

Of course, this requires that you explicitly maintain a list of each Task object, including those which have already completed. If you intend for this to run for a long time and process a very large number of symbols, that might be prohibitive. In that case, you might prefer to use the CountDownEvent object:

static void Main(string[] args)
{
    CountDownEvent countDown = new CountDownEvent();

    while (true) 
    {
        Console.Write("Enter a symbol to research or [ENTER] to exit: ");
        string symbol = Console.ReadLine();
        if (string.IsNullOrEmpty(symbol))
            break;

        countDown.AddCount();
        DownloadDataForStockAsync(symbol).ContinueWith(task => countdown.Signal()) ;
    }

    countDown.Wait();
}

This simply increments the CountDownEvent counter for each task you create, and attaches a continuation to each task to decrement the counter. When the counter reaches zero, the event is set, allowing a call to Wait() to return.

这篇关于使用HttpClient的异步文件下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！