如何快速高效地将文件大块(html5块的结果)合并到一个文件中 [英] how to merge chunks of file (result of html5 chunking) into one file fast and efficient

问题描述

我创建了一个文件传输程序，该程序使用html5分块上传文件(大约4gb的大文件).每个块的大小为100MB(我无缘无故选择它，因为我尝试使用10MB，据我所知，它实际上并没有任何区别).

I create a file transfer program which upload files (huge file about 4gb) using html5 chunking. each chunk is sized of 100MB (I just choose this for no reason, as I try use 10MB, it does not really have any difference as far as I can tell).

它正确上传了每个块.但是在完成上传的最后，我尝试将文件合并回1件，但是要花很多时间.如果我尝试刷新上传者的Web ui，则在完成合并之前，它将无法正常工作.

It upload correctly each chunk. but at the end of finish uploading, I try to merge the file back into 1 piece, but it takes so much time. If I try to refresh the web ui for the uploader, it won't work until it finish merging.

我的合并代码如下:

$final_file_path = fopen($target_path.$file_name, "ab");


//Reconstructed File
for ($i = 0; $i <= $file_total_chunk; $i++) {
    $file_chunk = $target_path.$file_name.$i;    

    if ( $final_file_path ) {
        // Read binary input stream and append it to temp file
        $in = fopen($file_chunk, "rb");
        if ( $in ) {
            //while ( $buff = fread( $in, 1048576 ) ) {
            while ( $buff = fread( $in, 104857600 ) ) {
                fwrite($final_file_path, $buff);
            }   
        }
        if(fclose($in)) {
            unlink($file_chunk);
        }        
    }
}

fclose($final_file_path);   

无论如何，有效率和快速地做到这一点.我正在使用PHP.

Is there anyway to do it efficiently and fast. I'm using PHP.

谢谢

推荐答案

如果在使用带有 exec 函数的php时不想等待，可以使用

If you dont want to wait when using php with exec function, you can use gearman work queue with asynchronous response from workers. Inside worker you can use @hafichuk solution. Queue make your whole application more scalable.

这篇关于如何快速高效地将文件大块(html5块的结果)合并到一个文件中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！