如何同时发送多个 http 请求? [英] How do I send many http requests at the same time?

问题描述

我有一个 eBay 帐户.为了方便地更新我的产品，我用 C# 创建了一个 Windows 服务.它等待(通过 FileSystemWatcher)一个 xml 文件，一旦文件出现，服务就会读取它并通过他们的 API 向 eBay 服务器发送请求.一个文件可能包含大约 1000-3000 行.

I have an eBay account. To make updating my products easily, I created a windows service in C#. It waits (by FileSystemWatcher) for an xml file and once a file is appeared, a service reads it and sends requests to eBay server through their API. A file might hold about 1000-3000 rows.

以前我曾经创建 12 个线程以使其更快.我不知道为什么我选择了 12 个线程，我认为这个数量已经足够了:不要太多也不要太少.

Previously I used to create 12 threads to make it faster. I don't know why I chose 12 threads, I thought that was enough amount: not too much and not too little.

所以这是我之前的一个方法(有点脏，12应该是一个常量)

So here is a method how I looked like before (bit dirty, 12 should be a constant)

private static void UpdateAsync(IEnumerable<ProductInfo> products)
        {
            ProductInfo[] productsInfo = items.ToArray();
            Thread[] threads = new Thread[12];
            bool sendPartialy = productsInfo .Length > 12;
            int delta = 0;
            for (int i = 0; i < productsInfo.Length; i++)
            {
                if (sendPartialy )
                {
                    if (i != 0 && i % 12 == 0)
                    {
                        WaitForExecutedThreads(threads);
                        delta = 12 * (i / 12);
                    }
                }

                ProductInfo product = ProductInfo[i];
                threads[i - delta] = new Thread(_ => product.UpdateItem(context));
                threads[i - delta].Start();
            }

            WaitForExecutedThreads(threads);
        }

然后他们告诉我没有必要使用线程，因为只有一个网络接口:允许同时执行 12 个 https 请求的范围很窄，因此每个线程都会等待另一个.

Then they told me that using threads are not necessary because of only one network interface: it's narrow to allow 12 https requests be executed at the same time and thus, each of a thread will be waiting for another one.

这就是为什么我决定根本不使用多线程而只使用这样的简单请求:

That's why I decided not to use multi-threading at all and just use a simple requests like that:

    var itemInfoLights = new List<ItemInfoLight>();
    foreach (var p in productsInfo )
    {
       p.UpdateItem(context);
    }

我知道，单独发送所有请求太糟糕了，因为每个请求都有自己的标头等......这效率不高.

I know, it's too bad to send all request separately because each of them has their own headers, etc... That's not efficient.

那么，这样做的正确方法是什么?

Well, what is the right approach to do that?

推荐答案

由于您在此网络操作期间不需要 CPU，因此创建新线程并不是最佳选择.(当 CPU 是实际问题时，通常每个处理器只需要 1 个线程.因此，2 或 4 个线程是理想的.)

Since you don't need the CPU during this network operation, creating new threads isn't the optimal choice. (When CPU is the actual issue, you generally want only 1 thread per processor. So, 2 or 4 threads would be ideal.)

如果您有 UpdateItem 方法的异步版本，您可以根据需要更新的项目多次调用它，然后在回调函数中处理完成.这会更有效率，因为您不会启动实际上不做任何事情的线程，而是几乎同时执行多个请求.

If you had an async version of the UpdateItem method, you could call it as many times as you have items to update, then handle the completion in a callback function. This would be much more efficient, since you would not be starting threads that don't really do anything, but you would be executing multiple requests at nearly the same time.

如果您没有该方法的异步版本，使用线程将是获得并行更新调用的唯一简单方法.您可以使用 ThreadPool.QueueUserWorkItem 代替.这利用了已经可用的线程，避免了每次启动新线程的成本.

In the event that you don't have an async version of the method, using threads will be the only easy way to get side-by-side update calls. You might use ThreadPool.QueueUserWorkItem instead. That takes advantage of threads which are already available, avoiding the cost of spinning up new threads every time.

这篇关于如何同时发送多个 http 请求?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！