创建并行libcurl HTTP请求 [英] Make parallel libcurl HTTP requests

问题描述

我有一个关于使用libcurl（C ++）执行并行HTTP请求的安全性的问题。在阅读这个问题时，请记住我对一般的HTTP请求的了解有限。

基本上，我有两个（或更多）线程，每个线程每秒进行一次HTTP请求。 （所有请求都发送到同一台服务器）。我的程序（或别的什么？）跟踪什么HTTP响应属于什么胎面？我的意思是，我可以确保如果请求A是从线程1发出的，并且同时从线程2请求B，并且响应同时被取回，正确的响应（响应A）去往线程1，对线程2的响应B？

请原谅我的无知。

解决方案

首先，libcurl是线程安全：

libcurl设计和实现完全线程安全

正如本官方文档所指出的，您需要做的是：

从不在多个线程之间共享libcurl句柄。

此外，还有一个官方常见问题条目，例如如果您计划使用SSL：

如果在多线程环境中使用基于OpenSSL的libcurl，您需要提供一个或两个锁定函数

< blockquote>

正如你所看到的，有一个官方示例说明了多线程使用简单句柄：参见 multithread.c ：

  / *这是当新线程启动时执行的回调* / 
 static void * pull_one_url（void * url）
 {
 CURL * curl; 
 
 curl = curl_easy_init（）; 
 curl_easy_setopt（curl，CURLOPT_URL，url）; 
 curl_easy_perform（curl）; / *忽略错误* / 
 curl_easy_cleanup（curl）; 
 
 return NULL; 
} 
 
 / * ... * / 
 
 / *这是创建用于并行提取的NUMT线程的主循环* / 
 for（i = 0; i  error = pthread_create（& tid [i]，
 NULL，/ * default attributes please * / 
 pull_one_url，
（void *）urls [i]）; 
 / * ... * / 
} 
  

这个例子是一个尝试。

最后记住，libcurl还提供了一个所谓的 multi interface 其中使用单个线程提供多个传输。

编辑

关于OpenSSL +多线程，有可能有帮助的具体官方示例：

• opensslthreadlock.c


• threaded-ssl.c

I have a question regarding the safety of performing parallel HTTP-requests using libcurl (C++). When reading this question, please bear in mind I have limited knowledge about HTTP-requests in general.

Basically, let's say I have two (or more) threads, each thread makes a HTTP-request once per second. (All the requests made are to the same server). How does my program (or something else?) keep track of what HTTP-response belongs to which tread? I mean, can I be sure that if request A was sent from thread 1, and request B from thread 2 at the same time, and the responses are retrived at the same time, the correct reponse (response A) goes to thread 1 and response B to thread 2?

Please excuse my ignorance in this matter.

Thanks.

解决方案

First of all libcurl is thread safe:

libcurl is designed and implemented entirely thread safe

As pointed out by this official documentation all you need to do is:

Never share libcurl handles between multiple threads. You should only use one handle in one single thread at any given time.

In addition there is also this official FAQ entry that provides some more precisions, e.g if you plan to use SSL:

If you use a OpenSSL-powered libcurl in a multi-threaded environment, you need to provide one or two locking functions

As you can see there is an official example which illustrates a multi-threaded use of the easy handles: see multithread.c:

/* This is the callback executed when a new threads starts */
static void *pull_one_url(void *url)
{
  CURL *curl;

  curl = curl_easy_init();
  curl_easy_setopt(curl, CURLOPT_URL, url);
  curl_easy_perform(curl); /* ignores error */ 
  curl_easy_cleanup(curl);

  return NULL;
}

/* ... */

/* This is the main loop that creates `NUMT` threads for parallel fetching */
for(i=0; i< NUMT; i++) {
    error = pthread_create(&tid[i],
                           NULL, /* default attributes please */ 
                           pull_one_url,
                           (void *)urls[i]);
  /* ... */
}

So feel free to give this example a try.

At last keep in mind that libcurl also provides a so-called multi interface which offers multiple transfers using a single thread. You may find it convenient too according to your use case.

EDIT

Regarding OpenSSL + multi-threads there are specific official examples that might help:

• opensslthreadlock.c
• threaded-ssl.c

这篇关于创建并行libcurl HTTP请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！