如何在没有内存开销的情况下在 PHP curl 中发布大量数据? [英] How to POST a large amount of data within PHP curl without memory overhead?

问题描述

我目前正在使用 PHP curl 扩展来与一些 HTTP API 通信.

我使用批量加载器一次执行大量操作.批量端点必须使用 POST 方法调用，所以我使用如下代码:

批量端点允许我发送大量数据(一次超过 200Mo).目前我需要将数据加载到一个需要 PHP 能够使用足够内存的变量中...

我需要为批量加载将 memory_limit 设置为一个较高的值...

有没有办法使用文件流发送带有 curl PHP 扩展的数据?我看到了 CURLOPT_INFILE 和 CURLOPT_READFUNCTION 但它似乎不适用于 POST 方法...

我还看到 curl 命令行工具能够执行 --data "@/path/to/file/content" 这似乎是我需要的...

有什么想法吗?

解决方案

使用 CURLOPT_INFILE

$curl = curl_init();curl_setopt( $curl, CURLOPT_PUT, 1 );curl_setopt( $curl, CURLOPT_INFILESIZE, filesize($tmpFile) );curl_setopt( $curl, CURLOPT_INFILE, ($in=fopen($tmpFile, 'r')) );curl_setopt( $curl, CURLOPT_CUSTOMREQUEST, 'POST' );curl_setopt( $curl, CURLOPT_HTTPHEADER, [ 'Content-Type: application/json' ] );curl_setopt( $curl, CURLOPT_URL, $url );curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );$result = curl_exec($curl);curl_close($curl);fclose($in);

I'm currently using PHP curl extension to communicate with some HTTP APIs.

I use a bulk loader to perform a lot of operations at once. The bulk endpoint must be called with POST method so I use a code like :

<?php
$h = curl_init($url);
curl_setopt(CURLOPT_POST, true);
curl_setopt(CURLOPT_POSTFIELDS, $data);
curl_setopt(CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($h);
curl_close($h);

The bulk endpoint allows me to send a huge amount of data (more than 200Mo at once). For the moment I need to load the data inside a variable which require PHP to be able to use enough memory...

I need to set memory_limit to a high value just for the bulk load...

Is there a way to use a file stream to send data with the curl PHP extension ? I saw the CURLOPT_INFILE and CURLOPT_READFUNCTION but it seems to don't work with POST method...

I also saw that the curl command line tool is able to perform a --data "@/path/to/file/content" which seems to be what I need...

Any ideas ?

解决方案

Use CURLOPT_INFILE

$curl = curl_init();
curl_setopt( $curl, CURLOPT_PUT, 1 );
curl_setopt( $curl, CURLOPT_INFILESIZE, filesize($tmpFile) );
curl_setopt( $curl, CURLOPT_INFILE, ($in=fopen($tmpFile, 'r')) );
curl_setopt( $curl, CURLOPT_CUSTOMREQUEST, 'POST' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, [ 'Content-Type: application/json' ] );
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );
$result = curl_exec($curl);
curl_close($curl);
fclose($in);

这篇关于如何在没有内存开销的情况下在 PHP curl 中发布大量数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！