C＃-如何同时执行多个Web请求 [英] C# - how to do multiple web requests at the same time

问题描述

我编写了一个代码来检查url，但是，ir的工作速度确实很慢。.我想尝试使其同时在少数几个url上工作，例如10个url，或者至少使其尽可能快。 / p>

我的代码：

  Parallel.ForEach（urls，new ParallelOptions { 
 MaxDegreeOfParallelism = 10 
}，s => {
 try {
 using（HttpRequest httpRequest = new HttpRequest（））{
 httpRequest.UserAgent = Mozilla / 5.0（Windows NT 10.0; WOW64; rv：52.0）Gecko / 20100101 Firefox / 52.0; 
 httpRequest.Cookies = new CookieDictionary（false）; 
 httpRequest.ConnectTimeout = 10000; 
 httpRequest。 ReadWriteTimeout = 10000; 
 httpRequest.KeepAlive = true; 
 httpRequest.IgnoreProtocolErrors = true; 
字符串检查= httpRequest.Get（s +'，null）.ToString（）; 
 if（errors.Any（new Func< string，bool>（check.Contains）））{
 Valid.Add（s）; 
 Console.WriteLine（s）; $ b $ F ile.WriteAllLines（Environment.CurrentDirectory + /Good.txt，有效）; 
} 
} 
}捕获{
 
} 
}）； 
  

解决方案

您的服务调用不太可能是 CPU绑定。因此，分配更多线程来处理负载可能不是最好的方法-如果使用 async 和 await ，如果可以的话，请使用更现代的 HttpClient 而不是HttpRequest或HttpWebRequest。

以下是操作示例：

  var client = new HttpClient（ ）; 
 
 //以网址列表开头
 var urls =新字符串[] 
 {
 http://www.google.com，
 http://www.bing.com； 
}; 
 
 //启动所有请求
 var request = urls选择
（
 url => client.GetAsync（url）
） .ToList（）; 
 
 //等待所有请求完成
等待Task.WhenAll（requests）; 
 
 //获取响应
 var response = request.Select 
（
 task => task.Result 
）; 
 
 foreach（响应中的var r）
 {
 //提取消息正文
 var s =等待r.Content.ReadAsStringAsync（）; 
 Console.WriteLine（s）; 
} 
  

I wrote a code to check urls, however, ir works really slow.. I want to try to make it work on few urls at the same time, for example 10 urls or at least make it as fast as possible.

my Code:

Parallel.ForEach(urls, new ParallelOptions {
  MaxDegreeOfParallelism = 10
}, s => {
  try {
    using(HttpRequest httpRequest = new HttpRequest()) {
      httpRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0";
      httpRequest.Cookies = new CookieDictionary(false);
      httpRequest.ConnectTimeout = 10000;
      httpRequest.ReadWriteTimeout = 10000;
      httpRequest.KeepAlive = true;
      httpRequest.IgnoreProtocolErrors = true;
      string check = httpRequest.Get(s + "'", null).ToString();
      if (errors.Any(new Func < string, bool > (check.Contains))) {
        Valid.Add(s);
        Console.WriteLine(s);
        File.WriteAllLines(Environment.CurrentDirectory + "/Good.txt", Valid);
      }
    }
  } catch {

  }
});

解决方案

It is unlikely that your service calls are CPU-bound. So spinning up more threads to handle the load is maybe not the best approach-- you will get better throughput if you use async and await instead, if you can, using the more modern HttpClient instead of HttpRequest or HttpWebRequest.

Here is an example of how to do it:

var client = new HttpClient();

//Start with a list of URLs
var urls = new string[]
    {
        "http://www.google.com",
        "http://www.bing.com"
    };

//Start requests for all of them
var requests  = urls.Select
    (
        url => client.GetAsync(url)
    ).ToList();

//Wait for all the requests to finish
await Task.WhenAll(requests);

//Get the responses
var responses = requests.Select
    (
        task => task.Result
    );

foreach (var r in responses)
{
    // Extract the message body
    var s = await r.Content.ReadAsStringAsync();
    Console.WriteLine(s);
}

这篇关于C＃-如何同时执行多个Web请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！