为大文件转换Uint8Array崩溃浏览器 [英] Converting Uint8Array crashing browser for large files
问题描述
有一个应用程序，其中输入类型为文件。以下方法获取文件，然后准备通过AJAX将其发送到服务器。
Have an app where there is an input of type "file". The following methods grab the file, then prep it to be sent to the server via AJAX.
private StartUpload = (files) => {
if (files && files.length === 1) {
this.GetFileProperties(files[0])
.done((properties: IFileProperties) => {
$('input[type=file]').val("");
if (this._compatibleTypes.indexOf(properties.Extension) >= 0) {
var base64 = this.ArrayBufferToBase64(properties.ArrayBuffer);
this.DoFileUpload(base64, properties.Extension).always(() => {
this.ShowDialogMessage('edit_document_upload_complete', 'edit_document_upload_complete');
});
} else {
this.ShowDialogMessage('edit_document_upload_incompatible', 'edit_document_upload_compatible_types', this._compatibleTypes);
}
});
} else {
this.ShowDialogMessage('edit_document_upload_one_file', 'edit_document_upload_one_file_msg');
}
};
private ArrayBufferToBase64(buffer): any {
var binary = '';
var bytes = new Uint8Array(buffer);
for (var xx = 0, len = bytes.byteLength; xx < len; xx++) {
binary += String.fromCharCode(bytes[xx]);
}
return window.btoa(binary);
}
private DoFileUpload = (base64, extension) => {
this.IsLoading(true);
var dfd = $.Deferred();
var data = {
data: base64
};
UpdateFormDigest((<any>window)._spPageContextInfo.webServerRelativeUrl, (<any>window)._spFormDigestRefreshInterval);
var methodUrl = "_vti_bin/viewfile/FileInformation.asmx/AddScannedItemAlt";
$.ajax({
headers: {
"X-RequestDigest": $("#__REQUESTDIGEST").val()
},
url: methodUrl,
contentType: "application/json",
data: JSON.stringify(data),
dataType: 'json',
type: "POST",
success: (response) => {
// do stuff
},
error: (e) => {
// do stuff
}
});
return dfd;
};
这绝对适用于绝大多数情况。但是，当文件大小很大（比如200MB +）时，它会杀死浏览器。
This works perfectly in the vast majority of cases. However, when the file size is large (say 200MB+) it kills the browser.
-
Chrome显示一个带有aw snap消息的黑灰色页面，基本上已经死了。
Chrome shows a blackish-grey page with the "aw snap" message and basically dies.
IE显示Out of Memory控制台错误但仍继续有效。
IE shows an "Out of Memory" console error but continues to work.
FF显示无响应脚本警告。选择不再显示我让它运行直到出现内存不足控制台错误。
FF shows an "Unresponsive script" warning. Choosing "don't show me again" lets it run until an "out of memory" console error shows up.
这就是它死的地方：
for (var xx = 0, len = bytes.byteLength; xx < len; xx++) {
binary += String.fromCharCode(bytes[xx]);
}
围绕此尝试/捕获并没有任何作用，也没有发现错误。
Wrapping a try/catch around this does nothing and no error is caught.
我可以在没有崩溃的情况下进入循环，但是因为len = 210164805，所以逐步完成每次迭代都很困难。为此，我尝试将console.log（xx）添加到循环并让它飞 - 但浏览器在日志中显示任何内容之前崩溃。
I can step into the loop without a crash, but stepping through every iteration is tough since len = 210164805. For this I tried to add console.log(xx) to the loop and let it fly - but the browser crashes before anything shows up in the log.
字符串可能导致浏览器的大小是否有限制一旦超过崩溃？
Is there some limit to the size a string can be that could be causing the browser to crash once exceeded?
谢谢
推荐答案
你需要做什么这可以通过在块或时间段中分解代码以异步方式进行。
You need to do this in an asynchronous way by breaking up the code either in blocks or time segments.
这意味着您的代码需要使用回调，否则它是直接的 -
This means your code will need to use callback, but otherwise it's straight forward -
var bytes = new Uint8Array(256*1024*1024); // 256 mb buffer
convert(bytes, function(str) { // invoke the process with a callback defined
alert("Done!");
});
function convert(bytes, callback) {
var binary = "", blockSize = 2*1024*1024, // 2 mb block
block = blockSize, // block segment
xx = 0, len = bytes.byteLength;
(function _loop() {
while(xx < len && --block > 0) { // copy until block segment = 0
binary += String.fromCharCode(bytes[xx++]);
}
if (xx < len) { // more data to copy?
block = blockSize; // reinit new block segment
binary = ""; // for demo to avoid out-of-memory
setTimeout(_loop, 10); // KEY: async wait
// update a progress bar so we can see something going on:
document.querySelector("div").style.width = (xx / len) * 100 + "%";
}
else callback(binary); // if done, invoke callback
})(); // selv-invoke loop
}
html, body {width:100%;margin:0;overflow:hidden}
div {background:#4288F7;height:10px}
<div></div>
使用转换为字符串的大缓冲区可能会使客户端内存不足。转换为字符串的200mb的缓冲区将增加2 x 200mb，因为字符串存储为UTF-16（即每个字符2个字节），所以这里我们使用600 mb开箱即用。
Using large buffers converted to string will possibly make the client run out of memory. A buffer of 200mb converted to string will add 2 x 200mb as strings are stored as UTF-16 (ie. 2 bytes per char), so here we use 600 mb out of the box.
这取决于浏览器以及它如何处理内存分配以及系统当然。浏览器将尝试保护计算机免受恶意脚本的攻击，这些脚本会尝试填充内存。
It depends on browser and how it deals with memory allocations as well as the system of course. The browser will try to protect the computer against malevolent scripts which would attempt to fill up the memory for example.
您应该能够留在ArrayBuffer并将其发送到服务器。
You should be able to stay in ArrayBuffer and send that to server.
这篇关于为大文件转换Uint8Array崩溃浏览器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
