在将文件写入客户端磁盘时从 blob 创建/对象 URL 中释放内存 [英] Releasing memory from blob creation / object URL in writing file to client's disk

问题描述

更新

自从提出下面的问题并在发现代码中的错误后得出一个更基本的问题后，我发现了更多信息，例如在 MDN 网络文档中下载 API 方法 downloads.download() 它指出撤销对象 url 应该只在 文件/url 下载后执行.因此，我花了一些时间试图了解网络扩展是否使下载 API onChanged 事件对网页的 javascript可用"，但我认为它不会.我不明白为什么下载 API 仅可用于扩展，尤其是当有很多关于相同内存使用/对象 URL 撤销问题的问题时.例如，等待用户完成在 Javascript 中下载一个 blob.

Since asking the question below and arriving at a more fundamental question after finding the error in the code, I found some more information such as in the MDN web docs for the downloads API method downloads.download() it states that a revoke of an object url should be performed only after the file/url has been downloaded. So, I spent some time trying to understand whether or not a web extension makes the downloads API onChanged event 'available' to javascript of a web page and don't think it does. I don't understand why the downloads API is available to extensions only, especailly when there are quite a few questions concerning this same memory-usage/object-url-revocation issue. For example, Wait for user to finish downloading a blob in Javascript.

如果你知道，请你解释一下?谢谢.

If you know, would you please explain? Thank you.

从关闭 Firefox 浏览器开始，右键单击本地 html 文件以在 Firefox 中打开，它打开时会在 Windows 任务管理器中看到五个 firefox.exe 进程.其中四个进程的内存在 20,000k 到 25,000k 之间，一个进程的内存约为 115,000k.

Starting with Firefox browser closed, and right clicking on a local html file to open in Firefox, it opens with five firefox.exe processes as viewed in Windows Task Manager. Four of the processes start with between 20,000k and 25,000k of memory and one with about 115,000k.

此 html 页面有一个 indexedDB 数据库，其中包含 50 个对象存储，每个存储包含 50 个对象.每个对象从其对象存储中提取并使用 JSON.stringify 转换为字符串，并写入二维数组.之后，数组的所有元素都连接成一个大字符串，转换为 blob 并通过 URL 对象写入硬盘，该对象随后立即被撤销.最终文件大约 190MB.

This html page has an indexedDB database with 50 object stores each containing 50 objects. Each object is extracted from its object store and converted to string using JSON.stringify, and written to a two-dimensional array. Afterward, all elements of the array are concatenated into one large string, converted to a blob and written to the hard disk through a URL object which is revoked immediately afterward. The final file is about 190MB.

如果代码在转换为 blob 之前停止，则 firefox.exe 进程之一的内存使用量增加到大约 425,000k，然后在数组元素被转换后大约 5-10 秒内回落到 25,000k连接成一个字符串.

If the code is stopped just before the conversion to blob, one of the firefox.exe process's memory usage increases to around 425,000k and then falls back to 25,000k in about 5-10 seconds after the elements of the array have been concatenated into a single string.

如果代码运行完成，同一个 firefox.exe 进程的内存使用量会增长到大约 1,000,000k，然后下降到大约 225,000k.从 115,000k 开始的 firefox.exe 进程也在代码的 blob 阶段增加到大约 325,000k，并且永远不会减少.

If the code is run to completion, the memory usage of that same firefox.exe process grows to about 1,000,000k and then drops to about 225,000k. The firefox.exe process that started at 115,000k also increases at the blob stage of the code to about 325,000k and never decreases.

在将 blob 作为文本文件写入磁盘后，这两个 firefox.exe 进程永远不会释放大约 2 x 200,000k 的内存增加.

After the blob is written to disk as a text file, these two firefox.exe processes never release the approximate 2 x 200,000k increase in memory.

我已将每个函数中使用的每个变量都设置为 null，除非刷新页面，否则永远不会释放内存.另外，这个过程是由一个按钮点击事件发起的；如果在没有中间刷新的情况下再次运行，这两个 firefox.exe 进程中的每一个都会在每次运行时额外占用 200,000k 内存.

I have set every variable used in each function to null and the memory is never freed unless the page is refreshed. Also, this process is initiated by a button click event; and if it is run again without an intermediate refresh, each of these two firefox.exe processes grab an additional 200,000k of memory with each run.

我一直无法弄清楚如何释放内存?

I haven't been able to figure out how to free the memory?

这两个函数很简单.json[i][j] 保存数据库中第 i 个对象存储中的第 j 个对象的字符串版本.os_data[] 是一个小对象数组 { "name" : objectStoreName, "count" : n }，其中 n 是存储中对象的数量.如果未调用 write_to_disk，则 build_text 功能似乎会释放内存.因此，问题似乎与 blob 或 url 有关.

The two functions are quite simple. json[i][j] holds the string version of the jth object from the ith object store in the database. os_data[] is an array of small objects { "name" : objectStoreName, "count" : n }, where n is the number of objects in the store. The build_text fuction appears to release the memory if write_to_disk is not invoked. So, the issue appears to be related to the blob or the url.

我可能忽略了一些明显的东西.感谢您提供任何指导.

I'm probably overlooking something obvious. Thank you for any direction you can provide.

我从 JavaScript: Create and save file 看到我有一个错误revokeObjectURL(blob) 语句.它不能撤销 blob，需要将 createObjectURL(blob) 保存到一个变量中，比如 url，然后撤销 url，而不是 blob.

I see from JavaScript: Create and save file that I have a mistake in the revokeObjectURL(blob) statment. It can't revoke blob, the createObjectURL(blob) needed to be saved to a variable like url and then revoke url, not blob.

这在大多数情况下都有效，并且在大多数情况下，上述两个 firefox.exe 进程都会释放内存.这给我留下了一个关于 url 撤销时间的小问题.

That worked for the most part and the memory is released from both of the firefox.exe processes mentioned above, in most cases. This leaves me with one small question about the timing of the revoke of the url.

如果撤销是为了释放内存，那么url是否应该在文件下载成功后撤销?如果撤销发生在用户单击确定下载文件之前，会发生什么?假设我点击按钮从数据库准备文件，准备好后浏览器打开下载窗口，但我等了一会儿，想着给文件命名或保存在哪里，不会撤销声明已经运行，但浏览器仍然持有"该网址，因为它将下载什么?我知道我仍然可以下载文件，但是撤销是否仍然释放内存?从我对这个例子的少量试验来看，它似乎不会在这种情况下发布.

If the revoke is what allows for the release of memory, should the url be revoked only after the file has been successfully downloaded? If the revoke takes place before the user clicks ok to download the file, what happens? Suppose I click the button to prepare the file from the database and after it's ready the browser brings up the window for downloading, but I wait a little while thinking about what to name the file or where to save it, won't the revoke statment be run already but the url is still 'held' by the browser since it is what will be downloaded? I know I can still download the file, but does the revoke still release the memory? From my small amount of experimenting with this one example, it appears that it does not get released in this scenario.

如果有一个事件在文件成功或不成功下载到客户端时触发，这不是应该撤销url的时间吗?在撤销 url 之前设置几分钟的超时会更好，因为我很确定没有事件表明下载到客户端已经结束.

If there was an event that fires when the file has either successfully or unsuccessfully been downloaded to the client, is not that the time when the url should be revoked? Would it be better to set a timeout of a few minutes before revoking the url, since I'm pretty sure there is not an event indicating download to client has ended.

我可能不了解这方面的一些基本知识.谢谢.

I'm probably not understanding something basic about this. Thanks.

function build_text() {

    var i, j, l, txt = "";

    for ( i = 1; i <=50; i++ ) {

         l = os_data[i-1].count;

         for  ( j = 1; j <= l; j++ ) {

              txt += json[i][j] + '
';

         }; // next j

    }; // next i


    write_to_disk('indexedDB portfolio', txt); 

    txt = json = null;

} // close build_text




function write_to_disk( fileName, data ) {  

    fileName = fileName.replace(".",""); 

    var blob = new Blob( [data], { type: 'text/csv' } ), elem;  


    if ( window.navigator.msSaveOrOpenBlob ) {

         window.navigator.msSaveBlob(blob, fileName);

    } else {

        elem = window.document.createElement('a');

        elem.href = window.URL.createObjectURL(blob);

        elem.download = fileName;        

        document.body.appendChild(elem);

        elem.click();        

        document.body.removeChild(elem);

        window.URL.revokeObjectURL(blob);

   }; // end if


   data = blob = elem = fileName = null;


} // close write_to_disk

推荐答案

我有点不明白这里的问题是什么...

I am a bit lost as to what is the question here...

但让我们试着回答，至少是一部分:

But let's try to answer, at least part of it:

首先让我们解释一下 URL.createObjectURL(blob) 的大致作用:

For a starter let's explain what URL.createObjectURL(blob) roughly does:

它会创建一个 blob URI，这是一个指向内存中的 Blob blob 的 URI，就像它位于可访问的地方(如服务器)一样.
该 blob URI 会将 blob 标记为垃圾收集器 (GC) 不可收集，只要它尚未被撤销，您就不必在您的脚本中维护对 blob 的实时引用，但您仍然可以使用/加载它.

It creates a blob URI, which is an URI pointing to the Blob blob in memory just like if it was in an reachable place (like a server).
This blob URI will mark blob as being un-collectable by the Garbage Collector (GC) for as long as it has not been revoked, so that you don't have to maintain a live reference to blob in your script, but that you can still use/load it.

URL.revokeObjectURL 然后将断开 blob URI 和内存中的 Blob 之间的链接.它不会直接释放blob占用的内存，它只会删除自己对GC的保护，[不再指向任何地方].
因此，如果您有多个 Blob URI 指向同一个 Blob 对象，则仅撤销一个不会破坏其他 Blob URI.

URL.revokeObjectURL will then break the link between the blob URI and the Blob in memory. It will not free up the memory occupied by blob directly, it will just remove its own protection regarding the GC, [and won't point to anywhere anymore].
So if you have multiple blob URI pointing to the same Blob object, revoking only one won't break the other blob URIs.

现在，只有在 GC 启动时才会释放内存，这仅由浏览器内部决定，当它认为是最佳时机时，或者当它看到它没有其他选择时(通常当它错过了内存空间).

Now, the memory will be freed only when the GC will kick in, and this in only decided by the browser internals, when it thinks it is the best time, or when it sees it has no other options (generally when it misses memroy space).

所以你没有看到你的内存立即被释放是很正常的，根据经验，我会说 FF 不关心使用大量内存，当它可用时，使得 GC kick 不很多时候，这有利于用户体验(GC 经常导致延迟).

So it is quite normal that you don't see your memory being freed up instantly, and by experience, I would say that FF doesn't care about using a lot of memory, when it is available, making GC kick not so often, whihc is good for user-experience (GCing often results in lags).

对于您的下载问题，确实，Web API 并没有提供一种方法来了解下载是成功还是失败，即使下载刚刚结束.
对于撤销部分，这真的取决于你什么时候做.
如果直接在点击处理程序中进行，那么浏览器还没有完成预取请求，所以当点击(下载)的默认动作发生时，不会有任何由 URI 链接的内容了.
现在，如果您在保存"提示之后撤销 Blob URI，浏览器将执行预取请求，因此可能能够自行标记 Blob 资源不应被清除.但我认为这种行为不受任何规范的约束，最好至少等待窗口的 focus 事件，此时资源的下载应该已经开始.

For your download question, indeed, web APIs don't provide a way to know if a download has been successful or failed, nor even if it has just ended.
For the revoking part, it really depends on when you do it.
If you do it directly in the click handler, then the browser won't have done the pre-fetch request yet, so when the default action of the click (the download) will happen, there won't be anything linked by the URI anymore.
Now, if you do revoke the blob URI after the "save" prompt, the browser will have done a pre-fetch request, and thus might be able to mark by itself that the Blob resource should not be cleared. But I don't think this behavior is tied by any specs, and it might be better to wait at least for the window's focus event, at which point the downloading of the resource should already have started.

const blob = new Blob(['bar']);
const uri = URL.createObjectURL(blob);
anchor.href = uri;
anchor.onclick = e => {
  window.addEventListener('focus', e=>{
    URL.revokeObjectURL(uri);
    console.log("Blob URI revoked, you won't be able to download it anymore");
  }, {once: true});
};

<a id="anchor" download="foo.txt">download</a>

这篇关于在将文件写入客户端磁盘时从 blob 创建/对象 URL 中释放内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！