通过Youtube API上传大型视频，导致内存不足 [英] Uploading large video via Youtube API causing Out of Memory

问题描述

我使用PHP通过直接上传到YouTube来发送视频。它适用于较小尺寸的视频，但尝试发送390 MB视频时，我得到以下错误：

I'm using PHP to send videos via Direct Upload to Youtube. It works fine for smaller sized videos but when trying to send a 390 MB video, I get the following error:

PHP致命错误：的内存
（已分配3932160）（尝试分配
390201902字节）

PHP Fatal error: Out of memory (allocated 3932160) (tried to allocate 390201902 bytes)

code> memory_limit 但不起作用。

I've tried increasing memory_limit but that does not help.

    if ($isFile) {
        ini_set('memory_limit', '2G')
        $data = file_get_contents($data);
    }

    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

    $out = curl_exec($ch);
    curl_close($ch);

    return $out;

我也试过通过 exec $ c>，但是更糟糕的事情发生：

I've also tried running curl through exec() but then even weirder things happen:

curl
http://uploads.gdata.youtube.com/feeds/api/users/default/uploads
-H'POST / feeds / api / users / default / uploads
HTTP / 1.1'-H'主机：
uploads.gdata.youtube.com'-H
'授权：OAuth [ snip oauth
info]'-H'GData-Version：2'-H
'X-GData-Client：www.mywebsite.com'
-H'X-GData-Key ：key = [snip]'-H'Slug：video.AVI'-H'Content-Type：
multipart / related;
boundary =iUI5C0hzisAHkx9SvaRJ'-H
'Content-长度：390193710'-H
'连接：close'-d
/tmp/youtube.xml

curl http://uploads.gdata.youtube.com/feeds/api/users/default/uploads -H 'POST /feeds/api/users/default/uploads HTTP/1.1' -H 'Host: uploads.gdata.youtube.com' -H 'Authorization: OAuth [snip oauth info]"' -H 'GData-Version: 2' -H 'X-GData-Client: www.mywebsite.com' -H 'X-GData-Key: key=[snip]' -H 'Slug: video.AVI' -H 'Content-Type: multipart/related; boundary="iUI5C0hzisAHkx9SvaRJ"' -H 'Content-Length: 390193710' -H 'Connection: close' -d /tmp/youtube.xml

/tmp/youtube.xml是我已保存要上传的数据文件的位置，可能此用法有误？

/tmp/youtube.xml is where I have saved the data file to upload. Perhaps this usage is wrong?

这将需要大约6分钟，正在发送，但是我得到一个空的回复：

This will take about 6 minutes so it looks like the file is being sent but then I get an empty reply:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed

 0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
 0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
 ...
 0     0    0     0    0     0      0      0 --:--:--  0:06:00 --:--:--     0
curl: (52) Empty reply from server

编辑：

我使用OAuth，所以我不能使用正常的PHP API库。我必须将包含在XML文件中的视频的二进制数据的XML文件上传为此处概述

I am using OAuth so I cannot use the normal PHP API library for this. I must upload an XML file with the video's binary data included in the XML file as outlined here

我找到了另一个有相同问题的人，他们提供了代码来读取和发送大块文件。然而，当尝试这种方法，Youtube将返回一个411页，说Content-Length头是必需的。我设置的内容长度头，这可能是一个错误。此方法使用 fsockopen（）函数，而不是cURL。 [实际上，再看一遍代码，我意识到我只是用\\\
而不是\r\\\
分隔标题。这可能是问题。我会尝试用回车]

I found another person with the same problem who had supplied code to read and send the file in chunks. However, when trying this method, Youtube would return a 411 page saying the "Content-Length" header is required. I am setting the content-length header so this could be a bug. This method uses the fsockopen() function instead of cURL. [Actually, looking at the code again, I realise I was just separating the headers with "\n" instead of "\r\n." That might be the issue. I will try with the carriage returns as well]

编辑2：

我认为\r \\\
工作，但现在与代码，我再次收到一个空的回复从Youtube。

I think the "\r\n" worked, but now with the code, I am again receiving an empty reply from Youtube.

任何Curl专家，可以帮助我得到这个工作？

Any Curl experts out there that can help me get this working? I'm completely stumped by this.

推荐答案

尝试在发送之前不要将整个文件读入内存。 curl IMHO支持在上传之前读取文件本身，因此应该在内存边界内工作。有关示例，请参阅以下博文： http://dtbaker.com.au/random-bits/uploading-a-file-using-curl-in-php.html

Try not to read the whole file into memory before sending. curl IMHO supports reading the file itself before uploading and therefor should work inside the memory borders. See the following blog post for an example: http://dtbaker.com.au/random-bits/uploading-a-file-using-curl-in-php.html

我没有工作与YouTube直接上传API，但快速看看，我看到这似乎不是一个正常的HTML表单文件上传，但有点更复杂的数据格式。我不知道你是否可以用纯cURL做这些，而不用自己在内存中建立整个POST数据。

I haven't worked with the youtube direct upload API yet but after a quick look I saw that this does not seem the be a normal html form file upload but a bit more complex data format. I am not sure whether you can do this with plain cURL without building the whole POST data in memory yourself.

如果你有这么小的内存限制（〜4 MB的RAM ）你可以尝试在PHP的流API之上构建自己的简单的HTTP客户端：创建一个临时文件，并写入你的POST请求数据到该文件句柄（使用fwrite（）为正常字符串和stream_copy_to_stream（）直接文件到文件数据传输）。当您的请求准备就绪后，请将您的临时文件倒回开头，然后将该流复制到与youtube http服务器的连接中（再次使用stream_copy_to_stream（））。

If you have this small memory limits (~4 MB of RAM) you could try building your own simple HTTP client on top of the streams API in PHP: Create a temporary file and write your POST request data into that file handle (using fwrite() for normal strings and stream_copy_to_stream() for direct file to file data transfer). When your request is ready, rewind your temporary file to the beginning and then copy that stream into the connection with the youtube http server (again using stream_copy_to_stream()).

这些流被复制成小块，你应该能够做到这一点，少于4 MB的RAM，即使是大文件。

As the streams are copied in small chunks you should be able to do that with less then 4 MB of RAM, even for large files.

编辑：

下面的伪php-code-mashup应该是有帮助的：）

Following pseudo-php-code-mashup should be helpful ;)

$xmlAPIRequest = /* fill with the XML-API-Request */
$boundaryString = /* fill me with some random data */

// create temporary file handle
$pdh = tmpfile();
fwrite($pdh, "--$boundaryString\r\n");
fwrite($pdh, "Content-Type: application/atom+xml; charset=UTF-8\r\n\r\n");
fwrite($pdh, $xmlAPIRequest."\r\n");
fwrite($pdh, "--$boundaryString\r\n");
fwrite($pdh, "Content-Type: <video_content_type>\r\nContent-Transfer-Encoding: binary\r\n\r\n");

$videoFile = fopen("/path/to/video", "r");
stream_copy_to_stream($videoFile, $pdh);
fclose($videoFile);

fwrite($pdh, "--$boundaryString--\r\n");
/* not quite sure, whether there needs to be another linebreak before the boundary string */

$info = fstat($pdh);
rewind($pdh);

$contentLength = $info['size'];

$conn = fsockopen("hostname", 80);
/* write http request to $conn and use $contentLength for Content-Length header */
/* after last header you put another line break to tell them that now the body follows */

// write post data from stream to stream
stream_copy_to_stream($pdh, $conn);

// ... process response... etc...

在这段代码中肯定有很多错误，但它只是一个简单的例子，我认为我们可以生活在这。 ;）

There are certainly a lot of bugs in this code but as it is only a short example I think we can live with that. ;)

这篇关于通过Youtube API上传大型视频，导致内存不足的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！