在Web worker和主线程之间传递大量数据 [英] Pass large amounts of data between web worker and main thread

问题描述

有一种方法可以在web worker和主线程之间传递大量数据（多MB）吗？我在一个需要下载文件的项目中工作，稍微修改它们，然后以某种方式让用户下载修改后的文件。我发现了以下几种方式在Web工作者和主UI之间传递数据

1. 使用常规 postMessage
   code> method。


2. 使用可传输对象（仅限Chrome）


3. 创建一个URL引用并仅发送URL在Chrome中，在其他方面呢？）

我认为（1）发送较小的对象时很好，但需要很长时间并处理大于几MB的文件，因为它被序列化并作为JSON发送。 Chrome提供了一种使用可转移对象传输数据的方法，其中不必复制数据。不幸的是，这是迄今为止Chrome浏览器的唯一功能，因为它可以满足我的目的。

我发现的最后一件事是从工作人员创建一个URL到blob使用 self.webkitURL ，然后只将URL引用传递给UI。这很好，我可以给用户的URL，他们可以下载文件。我很遗憾没有找到一种方法来做到这一点是在Firefox，是否有可能？

是否有任何其他方法可用于在工作人员和主线程之间传输数据？

解决方案

Firefox / Opera / Chrome目前都支持一种叫做Transferable Objects的网络工作者味道，它非常快速 - 也非常容易设置。在这里，我们发送给ww（web worker）一个浏览器分配的数组，该数组由ww填充并返回到浏览器端。这是通过引用传递的，而不是副本：浏览器< - > ww

在浏览器端：

  var huge_array = new Float32Array（SIZE）; 
 
 // worker.postMessage（huge_array.buffer）; //旧方式
 worker.postMessage（huge_array.buffer，[huge_array.buffer]）; // new Trans Obj 
  

然后在web worker内部：

  self.onmessage = function（e）{
 
 var flt_arr = new Float32Array（e.data）; 
 
 //通常你可能想在这里填充flt_arr 
 
 //现在将数据结构发送回浏览器
 
 // self.postMessage（flt_arr 。缓冲）; //旧方式
 self.postMessage（flt_arr.buffer，[flt_arr.buffer]）; // new Trans Obj way 
} 
  

通过将数据对象放在方括号内[在这里]
这提示关闭js使用Transferable Object模式。这也适用于包含多个变量的JavaScript对象，而不仅仅是类型化的数组。

引用：

可转移对象是不被复制的对象（例如，使用类似结构化克隆的对象）。相反，数据从一个上下文传输到另一个上下文。调用上下文中的'版本'在传输到新的上下文后不再可用。例如，当将主应用程序中的ArrayBuffer传输给Worker时，主线程中的原始ArrayBuffer将被清除并且不再可用。这大大提高了向工作人员发送数据的性能

http://html5-demos.appspot.com/static/workers/transferables/index.html
https://html.spec.whatwg.org/multipage/workers.html

Is there a way to pass large amounts of data (multiple MB) between a web worker and the main thread? I work in a project where I need to download files, modify them a bit and then somehow let the user download the modified file. I found the following ways to pass data between a web worker and the main UI

1. Using the regular postMessage method.
2. Using transferable objects (Chrome only)
3. Create a URL reference to a blob and only send the URL (works in Chrome, in others as well?)

I think (1) is fine when sending smaller objects but it takes a lot of time and space when dealing with files larger than a few MB since it is serialized and sent as JSON. Chrome offers a way to transfer data using transferable objects where the data does not have to be copied. Unfortunately this is a Chrome only feature so far since it would have served my purpose otherwise.

The last thing I found was to, from the worker, create a URL to a blob using self.webkitURL and then only pass the URL reference to the UI. This works well, I can give the URL to the user and they can download the file. I have unfortunately not found a way to do this is in Firefox, is it possible?

Is there any other methods one could use to transfer data between workers and the main thread?

解决方案

Firefox/Opera/Chrome all currently support a flavor of web workers called Transferable Objects which is extremely fast - also extremely easy to setup. Here we send to the ww (web worker) a browser allocated array which is populated by the ww and returned back to the browser side. This is passed by reference, not a copy : browser <-> ww

On the browser side :

var huge_array = new Float32Array(SIZE);

// worker.postMessage(huge_array.buffer);                      // old way
   worker.postMessage(huge_array.buffer, [huge_array.buffer]); // new Trans Obj

and then over inside the web worker :

self.onmessage = function(e) {

      var flt_arr = new Float32Array(e.data);

    //  typically you might want to populate flt_arr here

    //  now send data structure back to browser

    // self.postMessage(flt_arr.buffer);                    // old way
       self.postMessage(flt_arr.buffer, [flt_arr.buffer]); // new Trans Obj way
}

By simply putting the data object inside square brackets [in here] this tips off js to use Transferable Object mode. This also works when sending back and forth javascript objects containing multiple variables, not just typed arrays.

To quote :

Transferable objects are objects that are not copied (e.g. using something like structured cloning). Instead, the data is transferred from one context to another. The 'version' from the calling context is no longer available once transferred to the new context. For example, when transferring an ArrayBuffer from main app to Worker, the original ArrayBuffer from the main thread is cleared and no longer usable. This vastly improves performance of sending data to a Worker

http://html5-demos.appspot.com/static/workers/transferables/index.html https://html.spec.whatwg.org/multipage/workers.html

这篇关于在Web worker和主线程之间传递大量数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！