哪个效率更高:多线程程序中的curl_easy_perform()或单线程程序中的curl_multi_perform()? [英] Which is more efficient: curl_easy_perform() in a multi-threaded program or curl_multi_perform() in a single threaded program?

问题描述

我正在开发一个程序，需要从不同的URL下载大量JSON文件.

I am working on a program where I am required to download a large amount of JSON files from different URLs.

当前，我的程序创建了多个线程，并且在每个线程中，它调用LibCurl easy_perform()函数，但是我遇到了程序偶尔失败且错误为"double free"的问题..似乎是某种Heisenbug，但我已经能够在GDB中将其捕获，这确认了错误源自LibCurl(回溯).

Currently, my program creates multiple threads, and in each thread, it calls the LibCurl easy_perform() function but I am running into issues where the program fails occasionally with an error of "double free". It seems to be some sort of Heisenbug but I have been able to catch it in GDB which confirms the error originates in LibCurl (backtraced).

虽然我希望就所遇到的问题提出建议，但我的实际问题是:如果我更改代码结构以在一个线程上使用LibCurl Multi Interface而不是调用单个接口会更好些吗?跨多个线程?在一个之上使用一个之上的权衡是什么?

While I would love suggestions on the issue I am having, my actual question is this: Would it be better if I were to change the structure of my code to use the LibCurl Multi Interface on one thread instead of calling the single interface across multiple threads? What are the trade offs of using one over the other?

注意:更好"是指它更快且对CPU的负担更少?多接口是为此目的而设计的吗?

Note: By "better", I mean is it faster and less taxing on my CPU? Is it more reliable as the multi interface was designed for this?

据我了解，我有以下三个选择:

The three options I have as I understand it are these:

1)在单个线程中重用相同的easy_handle.无需重新建立连接即可使其更快.

1) Reuse the same easy_handle in a single thread. The connections wont need to be reestablished making it faster.

2)在每个单独的线程中调用curl_easy_perform().它们再次并行运行，从而使其更快.

2) Call curl_easy_perform() in each individual thread. They all run in parallel, again, making it faster.

3)在单个线程中调用curl_multi_perform().这是非阻塞的，所以我想所有文件都是并行下载的，这样可以更快?

3) Call curl_multi_perform() in a single thread. This is non-blocking so I imagine all of the files are downloaded in parallel making it faster?

以下哪个选项最省时?

推荐答案

curl_easy_perform正在阻止操作.这意味着，如果您在一个线程中运行，则必须顺序下载文件.在多线程应用程序中，您可以并行运行许多操作-这通常意味着更快的下载时间(如果速度不受网络或目标服务器的限制).

curl_easy_perform is blocking operation. That means if you run in in one thread you have to download files sequentially. In multithreaded application you can run many operations in parallel - this usually means faster download time (if speed is not limited by network or destination server).

但是，如果您想采用单线程方式，则有一个非阻塞变体可能更适合您-curl_multi_perform

But there is non-blocking variant that may work better for you if you want to go single threaded way - curl_multi_perform

来自卷发男人

您可以在使用一样的easy_handle.如果您打算传输多个文件，则您甚至被鼓励这样做.然后，libcurl将尝试重用后续传输使用相同的连接，因此操作更快，CPU占用更少，使用的网络资源也更少.请注意，您必须在两个之间使用curl_easy_setopt调用以设置以下curl_easy_perform的选项.

You can do any amount of calls to curl_easy_perform while using the same easy_handle. If you intend to transfer more than one file, you are even encouraged to do so. libcurl will then attempt to re-use the same connection for the following transfers, thus making the operations faster, less CPU intense and using less network resources. Just note that you will have to use curl_easy_setopt between the invokes to set options for the following curl_easy_perform.

简而言之-与curl_easy_perform相比，它没有什么好处.

In short - it will give few benefits you want vs curl_easy_perform.

这篇关于哪个效率更高:多线程程序中的curl_easy_perform()或单线程程序中的curl_multi_perform()?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！