如何使用AJAX上传大型CSV文件？ [英] How to use AJAX to upload large CSV file?
问题描述
我们有一些包含大量记录的CSV文件。需要使用Web界面将此文件上传到MySQL数据库。这些文件从不同的现场作品收集，并通过Web应用程序上传到服务器。 Web应用程序是使用PHP开发的。
最初，记录数非常少。
问题 -
- 我们必须等待上传整个文件才能开始处理。
- 上传完整个文件后，我们可以看到数据验证错误
- 默认情况下不支持上传大文件。
- PHP执行时间有限。
所以，我正在寻找一个解决方案来上传CSV文件内容作为块。它将帮助我处理每个请求的少量数据。
我的想法是使用JavaScript从CSV文件读取数据。
想要查看小数据量的数据验证错误，如果需要可以停止执行。
<在对Web进行一些研究之后，我发现HTML5 JavaScript File API对从本地文件系统读取内容非常有用。
在这个例子中，我创建了两个文件 -
- index.htm
- service.php
index.htm
<！DOCTYPE html& ; html> < head> < meta charset =UTF-8> < title>上传大型CSV文件< / title> < style> body {font-size：8pt; color：＃333} #wrap {width：500px; margin：5px auto} #responce {height：200px; overflow-y：scroll; border：1px solid #ddd;}< / style> < / head> < body> < div id =wrap> < ul id =responce> < / ul><！ - // response - > < input type =fileid =fileSelected/> < button id =btnUpload>上传< / button> < / div><！ - // wrap - > < script src =https://code.jquery.com/jquery-3.1.1.min.jsintegrity =sha256-hVVnYaiADRTO2PzUGmuLJr8BLUSjGIZsDYGmIJLv2b8 =crossorigin =anonymous>< / script> < script> var reader_offset = 0; //当前读取器偏移位置var buffer_size = 1024; // var pending_content =''; / ** *从当前偏移读取一个数据块。 * / function readAndSendChunk（）{var reader = new FileReader（）; var file = $（'＃fileSelected'）[0] .files [0]; reader.onloadend = function（evt）{//检查文件结束if（evt.loaded == 0）return; // increse offset value reader_offset + = evt.loaded; //检查只有完整的行var last_line_end = evt.target.result.lastIndexOf（'\\\
'）; var content = pending_content + evt.target.result.substring（0，last_line_end）; pending_content = evt.target.result.substring（last_line_end + 1，evt.target.result.length）; //上传数据send（content）; }; var blob = file.slice（reader_offset，reader_offset + buffer_size）; reader.readAsText（blob）; } / ** *使用AJAX发送数据到服务器* / function send（data_chank）{$ .ajax（{url：service.php，method：'POST'，data：data_chank} {//显示从服务器接收到的响应消息$（'#responce'）.append（'< li>'+ response +'< / li>'）; // try for next chank readAndSendChunk（）;}）; } / ** *在页面加载* / $（function（）{$（'＃btnUpload'）。click（function（）{reader_offset = 0; pending_content =''; readAndSendChunk（）;}）;}）; < / script> < / body>< / html>
service.php
<？php
$ content = file_get_contents（'php：// input '）;
$ lines = explode（\\\
，$ content）;
foreach（$ lines as $ line）{
$ csv_row = str_getcsv（$ line）;
//将数据保存到数据库
// ----
}
We have some CSV files with large number of records. Need to upload this files into a MySQL database using web interface. These files are collected from different field works and uploaded into the server through a web application. The web application is developed using PHP.
Initially the number of records was very small. But, now it's became very large and users unable to upload the file into the server.
Problems -
- We have to wait for upload the entire file to start processing.
- We can see the data validation errors after uploading the entire file.
- Uploading large files are not supported by default.
- PHP execution time is limited.
So, I am looking for a solution to upload CSV file content as chunk. It will help me to process a small amount of data per request.
My idea is to read data from CSV file using JavaScript. Then POST this data using AJAX.
Want to see the data validation errors against small amount of data and stop execution, if required.
After doing some research on web I have found HTML5 JavaScript File API is very useful to read content from local file system. I have written a small script and it's perfectly working.
For this example, I have created two files -
- index.htm
- service.php
index.htm
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Upload large CSV file</title>
<style>
body{font-size:8pt; color:#333}
#wrap{width:500px; margin:5px auto}
#responce{height:200px; overflow-y:scroll; border:1px solid #ddd;}
</style>
</head>
<body>
<div id="wrap">
<ul id="responce">
</ul><!-- // response -->
<input type="file" id="fileSelected"/>
<button id="btnUpload">Upload</button>
</div><!-- // wrap -->
<script
src="https://code.jquery.com/jquery-3.1.1.min.js"
integrity="sha256-hVVnYaiADRTO2PzUGmuLJr8BLUSjGIZsDYGmIJLv2b8="
crossorigin="anonymous"></script>
<script>
var reader_offset = 0; //current reader offset position
var buffer_size = 1024; //
var pending_content = '';
/**
* Read a chunk of data from current offset.
*/
function readAndSendChunk()
{
var reader = new FileReader();
var file = $('#fileSelected')[0].files[0];
reader.onloadend = function(evt){
//check for end of file
if(evt.loaded == 0) return;
//increse offset value
reader_offset += evt.loaded;
//check for only complete line
var last_line_end = evt.target.result.lastIndexOf('\n');
var content = pending_content + evt.target.result.substring(0, last_line_end);
pending_content = evt.target.result.substring(last_line_end+1, evt.target.result.length);
//upload data
send(content);
};
var blob = file.slice(reader_offset, reader_offset + buffer_size);
reader.readAsText(blob);
}
/**
* Send data to server using AJAX
*/
function send(data_chank)
{
$.ajax({
url: "service.php",
method: 'POST',
data: data_chank
}).done(function(response) {
//show responce message received from server
$( '#responce' ).append( '<li>' + response + '</li>');
//try for next chank
readAndSendChunk();
});
}
/**
* On page load
*/
$(function(){
$('#btnUpload').click(function(){
reader_offset = 0;
pending_content = '';
readAndSendChunk();
});
});
</script>
</body>
</html>
service.php
<?php
$content = file_get_contents('php://input');
$lines = explode("\n", $content);
foreach($lines as $line){
$csv_row = str_getcsv($line);
//save data into database
//----
}
这篇关于如何使用AJAX上传大型CSV文件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
