Jquery崩溃时解析大csv文件 [英] Jquery crashes while parsing large csv file
问题描述
我有一个jquery函数，我发现在一个教程网站，接受一个csv文件输入和tablifying它。我试图给一个大的csv文件（10,000KB），我的浏览器崩溃。我看到有一个称为papa的解析器库来处理这个，但是有没有其他方法来阻止我的浏览器崩溃，而这样做？
I have this jquery function which I found on a tutorial site that takes in a csv file input and tablifies it. I tried giving in a large csv file(10,000KB) and my browser crashes. I saw there's a parser library called papa to handle this but is there any other approach to keep my browser from crashing while doing this?
这里是相关的代码： -
Here's the relevant code:-
$("#upload").bind("click", function () {
var regex = /^([a-zA-Z0-9\s_\\.\-:])+(.csv|.txt)$/;
if (regex.test($("#fileUpload").val().toLowerCase())) {
var reader = new FileReader();
reader.onload = function (e) {
var table = $("<table id='mytable' class='table table-striped table-bordered'/>");
var rows = e.target.result.split("\n");
text1=e.target.result;
var csvString = $.trim(text1);
var csvRows = csvString.split(/\n/);
var csvHeaders = csvRows.shift().split(';');
var headerstr=String(csvHeaders);
var count = (headerstr.match(/,/g) || []).length+1;
for (var i = 0; i < rows.length; i++) {
var row = $("<tr />");
var cells = rows[i].split(",");
for (var j = 0; j < count; j++) {
if(cells[j]){
var cell = $("<td/>");
cell.html(cells[j]);
}
else{
var cell = $("<td class='info'/>");
cell.html("empty");}
row.append(cell);
}
table.append(row);
}
$("#dvCSV").html('');
$("#dvCSV").append(table);
}
});
如何在不损坏浏览器的情况下实现此功能？提前感谢。
How do I implement this functionality without crashing my browser? Thanks in advance.
推荐答案
解决这个问题的两个大问题：
Two big issues in tackling this problem:
1）CSV解析器。 Papa Parse 是伟大的。它支持工作，是一个流式语法分析程序 - 唯一的方式去处理大文件。
1) The CSV parser. Papa Parse is great. It has support for workers and is a streaming parser - the only way to go with large files.
2）显示数据的方式。简单地输出表中的每一行将不起作用。我摔坏了我的电脑两次，想出一个工作的解决方案。执行此和基本上任何处理大文件的系统的唯一方法是使用虚拟化列表。我最后使用这一个。
2) The way you display the data. Simply outputing each row in a table won't work. I crashed my PC twice trying to come up with a working solution. The only way of doing this and the one used by basically any system dealing with large files is to use virtualized lists. I ended up using this one. It's simple and the code is easy to understand.
这是我的JS：
$("#fUpload").bind("change", function(evt) {
var bigFile = evt.target.files[0];
var rows = [];
Papa.parse(bigFile, {
delimiter: ",",
newline: "\n",
header: false,
dynamicTyping: false,
worker: false,
chunk: function(results) {
rows.concat(rows, results.data);
},
complete: function() {
var list = new VirtualList({
h: 300,
itemHeight: 30,
totalRows: rows.length,
generatorFn: function(row) {
var el = document.createElement("div");
el.innerHTML = "<p>ITEM " + row + ' -> ' + rows[row].join(' - ') + "</p>";
return el;
}
});
document.body.appendChild(list.container)
}
});
});
HTML包含此输入：
< input type =文件id =fUpload/>
HTML contains this input:
<input type="file" id="fUpload" />
我如何配置Papa：
-
分隔符和换行符：如果允许它尝试和检测它们将失败或需要更长时间;
delimiterandnewline: if you allow it to try and detect them it will fail or take longer;
worker ：将生成一个工作进程。它会更慢，但会保持UI响应（UI线程不会做任何工作）。你可能想在生产中将它设置为 true 。 （由于浏览器的跨域安全协议，这不会在JSFiddle上工作！）;
worker: this will spawn a worker process. It will be slower but will keep the UI responsive (the UI thread won't do any work). You'll probably want to set this to true in production. (This won't work on JSFiddle because of browser cross-domain security protocol!);
chunk ：而不是每个解析行的回调有一个更大的一组行。这更快;
chunk: instead of a callback for each parsed row have one for a larger set of rows. This is faster;
虚拟列表配置是默认配置。
The virtual list config is the default one.
您可以在这里运行它。
我测试了一个9.4 MB的CSV文件，每行包含
1，Foo，100 。
I tested with a 9.4 MB CSV file containing
1,Foo,100 repeated on each line.
这里也是一样，但使用表格输出数据并添加了一个 -1 到VirtualList的 totalRows 以补偿实际长度。
Here's the same but using a table to output the data and added a -1 to VirtualList's totalRows to compensate for the actual length.
这篇关于Jquery崩溃时解析大csv文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
