在blob或文件中存储/检索大数组(字符串和类型数组的混合)的最佳方法 [英] Best way to store/retrieve big array (mixture of string and typed array) into/from blob or file

问题描述

假设我有一个大型数组，该数组具有不同类型的数据，例如 json字符串和类型化数组，例如mainData

Lets say I have big array which has different type of data like json string and typed array such as mainData

var strObj = JSON.stringify({'name' : 'suman', 'age' : 29};);
var strObj2 = JSON.stringify({'name' : 'laxmi', 'age' : 29});

var tpArr = new Uint8Array(2);
    tpArr[0]  = 42;
    tpArr[1]  = 52;

var tpArr2 = new Uint8Array(2);
    tpArr[0]  = 32;
    tpArr[1]  = 52;

var mainData = [{pt : 20, recObjs : strObj}, {pt : 30, recObjs : strObj2},
                {pt : 40, recObjs : tpArr}, {pt : 50, recObjs : tpArr2}];

我已经创建了数组为mainData的blob文件(类型为json)，并将其导出到json文件中，例如

I have created blob file(type json) with array mainData and export it into json file, like

var stringiFyData = JSON.stringify(mainData);   
var myBlob = new Blob([stringiFyData], {'type': 'application/json'});
var link2 = document.getElementById('mydata');
link2.href = window.URL.createObjectURL(myBlob);

现在，在将文件导出为json之后，我读取了类似的文件

Now after file exported as json, I read the file something like,

var fileInput = document.getElementById('fileInput');
var file = fileInput.files[0];
var reader = new FileReader();
reader.onload = function(e) {
     mainData = JSON.parse(reader.result);
}
reader.readAsText(file);    

现在mainData数组，tpArr和tpArr2的元素从 Uint8Array 丢失了其原始类型，而丢失到 Object 中.

Now elements of mainData array, tpArr, tpArr2 lost their's original type from Uint8Array into Object.

我需要原始格式的数据tpArr，tpArr2.为此，首先我将tpArr对象转换为数组.得到数组的长度后，创建类似

I need this data tpArr, tpArr2 in it's original form. For this, first I do convert tpArr object into array. After get the length of array, create typed array something like

newTypeArr = new Uint8Array(conNewArr.length);
for(var i=0; i<conNewArr.length; i++ ){
    newTypeArr[i] = conNewArr[i];
}

但是conNewArr数组的长度大约为3000，这意味着我们必须运行循环6000，第一个循环3000将对象转换为数组以知道长度，第二个3000为创建类型的数组.

But length of conNewArr array is about 3000 and which means we have to run loop 6000, first 3000 to convert object into array to know the length, and second 3000 to a create typed array.

当然，mainData可能具有成千上万种数组.

And of course the mainData may has that kind of array in thousands.

其他替代方法

我尝试将其导出(new Blob(mainData, {'type': 'application/octet-stream'})到数组缓冲区中，读取导出的文件后，我获得了所有类型化数组格式的数据，但是我需要这些数据，就像导出之前一样(字符串和类型化数组的混合)

I tried to export(new Blob(mainData, {'type': 'application/octet-stream'}) it into array buffer and after read the exported file, I got all the data in typed array format but I need these data as it was before exported(mixture of string and typed array).

我的问题是

1. 我们如何在blob文件中导出字符串和类型数组的mixtutre数据并从blob/文件中检索这些数据而不丢失其原始形式(类型)?
2. 我的方法还好吗?
3. 在我的方法中如何减少这种大循环的迭代次数?

window.URL = window.URL || window.webkitURL;
	var myObject = {'name' : 'suman', 'age' : 29};
	var myObject2 = {'name' : 'laxmi', 'age' : 29};
	
	var strObj = JSON.stringify(myObject);
	var strObj2 = JSON.stringify(myObject2);
	
	var tpArr = new Uint8Array(5);
		tpArr[0]  = 42;
		tpArr[1]  = 52;
		tpArr[2]  = 62;
		tpArr[3]  = 22;
		tpArr[4]  = 42;
	    
	var	tpArr2 = new Uint8Array(5);
		tpArr[0]  = 32;
		tpArr[1]  = 52;
		tpArr[2]  = 42;
		tpArr[3]  = 402;
		tpArr[4]  = 142;

	var mainData = [
		{pt : 20, recObjs : strObj},
		{pt : 30, recObjs : strObj2},
		{pt : 40, recObjs : tpArr},
		{pt : 50, recObjs : tpArr2}
		];
	
	
	var stringiFyData = JSON.stringify(mainData);	
	var myBlob = new Blob([stringiFyData], {'type': 'application/json'});
	var link2 = document.getElementById('mydata');
	link2.href = window.URL.createObjectURL(myBlob);
	
	function readFile (){
		var fileInput = document.getElementById('fileInput');
		var file = fileInput.files[0];
		var reader = new FileReader();
		reader.onload = function(e) {
			 data = JSON.parse(reader.result);
		}
		reader.readAsText(file);	
	}

<a href="#"	 id="mydata" >Download Data</a>
	<input type='file' id="fileInput" accept='text/plain' onchange='readFile()'>

推荐答案

为什么要导出所有Blob? 也许您想将所有数据导出为单个文件，然后在以后再导入.

Why do you want to export everything is Blob? Maybe you want to export all data as single file and import it again at later stage.

我建议关注.

最后将所有Typed数组转换为Base64和JSON.

Convert all Typed array into Base64 and JSON at end.

https://jsfiddle.net/esusedf7/2/

// Create a array with dummy Values
var typed_array = new Uint8Array(100000);
for (var i=0;i<100000;i++) {
    typed_array[i]=i;
}

// Convert Array into Base64
var sMyBase64 = base64EncArr(typed_array);
// Convert Base64 to Array
var ori1 = base64DecToArr(sMyBase64);

// Convert Array into String (comma seperated)
var comma = bufferToString(typed_array);
// Convert String (comma seperated) into Array
var ori2 = stringToBuffer(comma);

// Compare if converted array is exactly equal to Original Array
if (typed_array.length == ori2.length) {
    var match=0;
    for (var i=0;i<2000;i++) {
        if (typed_array[i] !== ori2[i]) {
            match = 1;
        }
    }
    if (match == 1) {
        alert('ori2 Fail');
    }
} else {
    alert('ori2 Fail');
}

原始数组:100000
Base64字符串:136844百分比增长137％
逗号字符串:356989百分比增加357％
詹森·斯金(Json String):1145881百分比增长1146％

Original Array: 100000
Base64 String: 136844 Percentage Increase 137%
Comma String: 356989 Percentage Increase 357%
Json String: 1145881 Percentage Increase 1146%

这篇关于在blob或文件中存储/检索大数组(字符串和类型数组的混合)的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！