在javascript中压缩blob [英] Compressing a blob in javascript
问题描述
我需要使用ajax发送一个blob到服务器，但它可能会变得有点大，我想减少上传时间。我已经尝试jszip，但只是给我一个空文件里面的zip。我也试过btoa（），但事实证明，编码的值最终是[object Blob]，而不是实际的blob数据。我的压缩blob的选项是什么？
以下是我用于jszip的代码：
var zip = new JSZip（）;
zip.file（recording.wav，blobFile）;
var content = zip.generate（）;
然后我将content附加到FormData对象并将其发送到服务器。在服务器端，我解码POST数据（从base64）。该zip文件打开正常，但recording.wav是一个长度为0的文件。
此外，我试过使用LZW实现找到在这里。这是我用来压缩它的附加代码：
var compressed;
var reader = new FileReader（）;
reader.onload = function（event）{
compressed = LZW.compress（event.target.result）;
};
reader.readAsText（blobFile）;
但是，解压缩会返回null。
注意：压缩诸如音频文件之类的东西最好使用专门用于这种类型数据的算法，或许是有损的。然而，知道如何很难找到一个合理的无损实现，如下所述，我很担心，很难找到一个良好的实现，在Javascript中的这种类型的数据，具体满足您的需求。
在任何情况下，我一直需要在JavaScript中压缩/解压缩，我需要相同的算法来工作客户端（浏览器）和服务器端（节点。 js），我需要它来处理非常大的文件。我检查了jszip和我也试过LZW算法至少五或六个人中没有一个满足要求。我不记得每个问题是什么具体，但足以说，令人惊讶的是，难以找到一个好的和FAST压缩器/解压缩在JavaScript中的工作，服务器和客户端和处理大文件！我尝试了至少十几种不同的压缩算法的实现，最后结算这一个 - 它没有失败我！
UPDATE
是原始来源：
https://code.google .com / p / jslzjb / source / browse / trunk / Iuppiter.js？r = 2
， 你是最好的。
它是LZJB： http://en.wikipedia.org/wiki/LZJB
UPDATE 2
- 修正了一个缺少分号的问题 - 不应该给对象不再是函数错误。
- 此实现停止处理长度小于约80个字符的数据。
- 实现的base64编码/解码方法实际上暴露在此版本的传入对象，因此...
- 目前看到我们可以做什么特定的Blob类型 - 例如最好的方法将是一个图像与音频等，因为这将是有用的JS人一般...将在这里更新与发现。
UPDATE 3
$ b
来自Bear的原始Iuppiter源代码比我在下面发布的代码有更好的包装。它是由cscott和github这里写的： https://github.com/cscott/lzjb
我将切换到这一个，因为它也流。
下面是一个使用Node.js的例子与wav文件。但在复制示例之前，让我给你可怕的消息，至少对于这一个wav文件，我尝试：
63128 Jun 19 14:09 beep-1.wav
63128 Jun 19 17:47 beep-2.wav
89997六月19 17:47 beep-2.wav.compressed
所以它成功地重新生成了wav（和它播放）。然而，压缩的一个似乎比原始的大。好的拍摄。在任何情况下，可能是好的尝试你的数据，你永远不知道，你可能会幸运。这里是我使用的代码：
var fs = require（'fs'）;
var lzjb = require（'lzjb'）;
fs.readFile（'beep-1.wav'，function（err，wav）{
// base 64 first
var encoded = wav.toString （'base64'）;
//然后utf8 - 你不想直接去utf-8
var data = new Buffer（encoded，'utf8'）;
// now compress
var compressed = lzjb.compressFile（data，null，9）;
//接下来的两行是不必要的，但是要看看什么样的
// size写入磁盘与原始二进制文件比较
var compressedBuffer = new Buffer（compressed，'binary'）;
fs.writeFile（'beep-2.wav.compressed'，compressedBuffer，'binary'，function ）{}）;
//解压缩
var uncompressed = lzjb.decompressFile（compressed）;
//从utf8回到base64
var encoded2 = new Buffer（uncompressed） .toString（'utf8'）;
//从base64解码回二进制原始数据
var decoded = new Buffer（encoded2，'base64'）;
//写出来它是相同的
fs.writeFile（'beep-2.wav'，decoded，function（err）{}）;
}）;
在一天结束时，我认为这将太难实现任何级别的压缩在大多数形式的二进制数据没有被生成的base64编码。终端的控制角色的日子仍然困扰我们这一天。你可以尝试到不同的基地，但它也有它的风险和问题，以及。
查看此示例：
什么是最有效的二进制文本编码？
有一件事，绝对在你接受答案之前，请尝试一下，使用它压缩utf-8，我想确保它对你的具体数据有效。
无论如何，这里是！
/ **
$ Id：Iuppiter.js 3026 2010-06-23 10：03：13Z Bear $
版权所有（c）2010 Nuwa Information Co.，Ltd和个人贡献者。
保留所有权利。
如果符合以下条件，则允许以源代码和二进制形式重新分发和使用（带或不带
修改）：
1.重新分发源代码必须保留上述版权声明，
此列表条件和以下免责声明。
2.以二进制形式再分发必须在
文档和/或分发提供的其他材料中重现上述版权
通知，此条件列表和以下免责声明。
3.在未经事先书面许可的情况下，Nuwa信息的名称或其贡献者的名称
不得用于支持或促销该软件衍生的产品
。
本软件由版权所有者和贡献者按原样提供
以及任何明示或暗示的保证，包括但不限于
对适销性的默示保证适用于特定目的是
免责声明。在任何情况下，版权所有者或捐赠者均不承担任何责任
任何直接，间接，偶发，特殊，惩罚或后果性损害赔偿（包括但不限于替代商品或购买
服务;使用，数据或利润损失;或业务中断）总额
由于责任原因，无论是合同，严格责任，
或侵权（包括疏忽或其他）任何方式使用本软件的
，即使已被告知此类损害的可能性。
$作者：Bear $
$日期：2010-06-23 18:03:13 +0800（星期三，23六月2010）$
$修订：3026 $
* /
var fastcompressor = {};
（function（k）{
k.toByteArray = function（c）{
var h = []，
b，a;
for（b = 0; b = a ?h.push（a> 12 | 224）：（h.push（a> 18 | 240），h.push（a> 63 | 128）），h.push（a> 6& 63 | 128）），h.push（a& 63 | 128））;
return h
};
k.Base64 = {
CA：ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 + /，
CAS：ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_，
IA：Array（256），
IAS：Array 256），
init：function（）{
var c;
for（c = 0; 256> c; c ++）k.Base64.IA [c] = - 1，k .Base64.IAS [c] = -1;
c = 0;
for（iS = k.Base64.CA.length; c
}，
encode：function（c，h）{
var b，a，d，e，m，g，f，l，j;
b = h？ k.Base64.CAS：k.Base64.CA;
d = c.constructor == Array？ c：k.toByteArray（c）;
e = d.length;
m = 3 *（e / 3）;
g =（e-1）/ 3 + 1 < 2;
a = Array（g）;
for（l = f = 0; f 18& 63），a [1 ++] = b.charAt（j> 12& 63），a [1 ++] = b.charAt （j> 6和amp; 63），a [1 ++] = b.charAt（j& 63）;
f = e - m;
0<费&&& （j =（d [m]&
255）<< 10 |（2 == f？（d [e-1]& 255）<2：0） g-4] = b.charAt（j> 12），a [g-3] = b.charAt（j> 6& 63），a [g-2] = 2 == f ？b.charAt（j& 63）：=，a [g-1] ==）;
return a.join（）
}，
decode：function（c，h）{
var b，a，d，e，m，g，f， l，j，p，q，n;
b = h？ k.Base64.IAS：k.Base64.IA;
c.constructor == Array？ （d = c，m = 0）：（d = k.toByteArray（c），m = 1）;
e = d.length;
g = 0;
for（f = e-1; g
l ==== d [f]？ === d [f-1]？ 2：1：0;
a = f-g + 1;
j = 76< e？ （\r== d [76]？a / 78：0）< 1：0;
e =（6 *（a-j）> 3）-1;
a = Array（e）;
q = p = 0;
for（eLen = 3 *（e / 3）; p
if（p
（b = 16; p b& 255
}
if（m）返回a;
for（n = 0; n
}
};
k.Base64.init（）;
NBBY = 8;
MATCH_BITS = 6;
MATCH_MIN = 3;
MATCH_MAX =（1 << MATCH_BITS）+（MATCH_MIN - 1）;
OFFSET_MASK =（1 << 16-MATCH_BITS）-1;
LEMPEL_SIZE = 256;
k.compress = function（c）{
var h = []，
b，a = 0，
d = 0，
e，m，g = 1< ;& NBBY - 1，
f，l，j = Array（LEMPEL_SIZE）;
for（b = 0; b 3435973836;
c = c.constructor == Array？ c：k.toByteArray（c）;
for（b = c.length; a< b;）{
if（（g <= 1）== 1 << NBBY）{
if d = b-1-2 * NBBY）{
f = b;
for（d = a = 0; f; f--）h [d ++] = c [a ++];
break
}
g = 1;
m = d;
h [d ++] = 0
}
if（a> b-MATCH_MAX）h [d ++] = c [a ++]
else if（e =（c [a] + 13 ^ c [a + 1] -13 ^ c [a + 2]）& LEMPEL_SIZE -1,1 = a-j [e]& OFFSET_MASK ，j [e] = a，e = a-l，0≤e& e！= a& c [a] == c [e]& ] == c [e + 1]& c [a + 2] == c [e + 2]）{
h [m]
for（f = MATCH_MIN; f
h [d ++] = l;
a + = f
} else h [d ++] = c [a ++]
}
return h
};
k.decompress = function（c，
h）{
var b，a = []，
d，e = 0，
m = 0，
g ，f，l = 1< NBBY - 1，
j;
b = c.constructor == Array？ c：k.toByteArray（c）;
for（d = b.length; e
if（j =（b [e]> NBBY- MATCH_BITS）+ MATCH_MIN，g =（b [e]
else break;
else a [m ++] = b [e ++]
}
if（！（undefined== typeof h？0：h））{
for（b = 0 ; b
}
return a
}
}）（fastcompressor）;
如果内存服务...这里是如何使用它：
var compressed = fastcompressor.compress（0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789）; //数据少于此长度会带来问题。
var decompress = fastcompressor.decompress（compressed）;
Rgds ... Hoonto / Matt
此外，我发布的是缩小但美化，非常轻微适应易于使用。请检查上述更新中的链接，了解原始内容。
I need to send a blob to the server with ajax, but it can end up getting somewhat large, and I'd like to decrease upload time. I've tried jszip already, but that just gave me an empty file inside the zip. I've also tried btoa(), but it turns out that the encoded value just ends up being [object Blob] instead of the actual blob data. What are my options for compressing blobs?
Here was the code I used for jszip:
var zip = new JSZip();
zip.file("recording.wav", blobFile);
var content = zip.generate();
I then appended "content" to a FormData object and sent it to the server. On the server side, I decoded the POST data (from base64). The zip file opened just fine, but recording.wav was a 0 length file.
Additionally, I've tried using the LZW implementation found here. This was the additional code I used to compress it:
var compressed;
var reader = new FileReader();
reader.onload = function(event){
compressed = LZW.compress(event.target.result);
};
reader.readAsText(blobFile);
However, decompressing it returns null.
Caveat: compressing things like audio files would be better done using an algorithm meant specifically for that type of data, perhaps something lossy. However, knowing how hard it was to find a reasonable lossless implementation as provided below, I'm very concerned that it will be hard to find a good implementation in Javascript for that type of data specifically that meets your needs.
In any case, I've had this general need for compression/decompression in Javascript as well, and I needed the same algorithm to work both client (browser) and server-side (node.js) and I needed it to work on very large files. I had checked out jszip and I also tried that LZW algorithm among at least five or six others none of which satisfied the requirements. I can't remember what the issue was with each specifically, but suffice to say it is surprisingly hard to find a good and FAST compressor/decompressor in javascript that works both server and client side and handles large files! I tried at least a dozen different implementations of various compression algorithms, and finally settled with this one - it hasn't failed me yet!
UPDATE
This is the original source: https://code.google.com/p/jslzjb/source/browse/trunk/Iuppiter.js?r=2
By someone named Bear - thanks Bear, whoever you are, you're the best. It is LZJB: http://en.wikipedia.org/wiki/LZJB
UPDATE 2
- Corrected a problem with missing semicolon - should not give the object not a function error any longer.
- This implementation stops working on data less than about 80 characters in length. So I updated the example to reflect that.
- Realized the base64 encode/decode methods are in fact exposed on the object passed in for this version, so...
- Currently seeing what we can do about specific blob types - what for example the best approach would be for a image versus audio etc as that would be useful for JS folks in general... will update here with what is found.
UPDATE 3
There is a much better wrapper around the original Iuppiter source from Bear than the one I posted below. It is written by cscott and on github here: https://github.com/cscott/lzjb
I'll be switching to this one, as it does streams as well.
Below is an example in Node.js of its use with a wav file. But before copying the example, let me give you the terrible news first, at least for this one wav file that I tried:
63128 Jun 19 14:09 beep-1.wav
63128 Jun 19 17:47 beep-2.wav
89997 Jun 19 17:47 beep-2.wav.compressed
So it successfully regenerated the wav (and it played). However, the compressed one appears to be larger than the original. Well shoot. In any case, might be good to try on your data, you never know, you might get lucky. Here's the code I used:
var fs = require('fs');
var lzjb = require('lzjb');
fs.readFile('beep-1.wav', function(err, wav){
// base 64 first
var encoded = wav.toString('base64');
// then utf8 - you don't want to go utf-8 directly
var data = new Buffer(encoded, 'utf8');
// now compress
var compressed = lzjb.compressFile(data, null, 9);
// the next two lines are unnecessary, but to see what kind of
// size is written to disk to compare with the original binary file
var compressedBuffer = new Buffer(compressed, 'binary');
fs.writeFile('beep-2.wav.compressed', compressedBuffer, 'binary', function(err) {});
// decompress
var uncompressed = lzjb.decompressFile(compressed);
// decode from utf8 back to base64
var encoded2 = new Buffer(uncompressed).toString('utf8');
// decode back to binary original from base64
var decoded = new Buffer(encoded2, 'base64');
// write it out, make sure it is identical
fs.writeFile('beep-2.wav', decoded, function(err) {});
});
At the end of the day, I think its going to be too difficult to achieve any level of compression on most forms of binary data that isn't clobbered by the resulting base64 encoding. The days of control characters for terminals still haunt us to this day. You could try upping to a different base, but that has its risks and issues as well.
See this for example: What is the most efficient binary to text encoding?
And this: Why don't people use base128 in JavaScript?
One thing though, definitely before you accept the answer, please please try it out on your blob, I've mainly used it for compressing utf-8, and I'd like to be sure it works on your specific data.
In any case, here it is!
/**
$Id: Iuppiter.js 3026 2010-06-23 10:03:13Z Bear $
Copyright (c) 2010 Nuwa Information Co., Ltd, and individual contributors.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of Nuwa Information nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
$Author: Bear $
$Date: 2010-06-23 18:03:13 +0800 (星期三, 23 六月 2010) $
$Revision: 3026 $
*/
var fastcompressor = {};
(function (k) {
k.toByteArray = function (c) {
var h = [],
b, a;
for (b = 0; b < c.length; b++) a = c.charCodeAt(b), 127 >= a ? h.push(a) : (2047 >= a ? h.push(a >> 6 | 192) : (65535 >= a ? h.push(a >> 12 | 224) : (h.push(a >> 18 | 240), h.push(a >> 12 & 63 | 128)), h.push(a >> 6 & 63 | 128)), h.push(a & 63 | 128));
return h
};
k.Base64 = {
CA: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/",
CAS: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_",
IA: Array(256),
IAS: Array(256),
init: function () {
var c;
for (c = 0; 256 > c; c++) k.Base64.IA[c] = -1, k.Base64.IAS[c] = -1;
c = 0;
for (iS = k.Base64.CA.length; c < iS; c++) k.Base64.IA[k.Base64.CA.charCodeAt(c)] = c, k.Base64.IAS[k.Base64.CAS.charCodeAt(c)] = c;
k.Base64.IA["="] = k.Base64.IAS["="] = 0
},
encode: function (c, h) {
var b, a, d, e, m, g, f, l, j;
b = h ? k.Base64.CAS : k.Base64.CA;
d = c.constructor == Array ? c : k.toByteArray(c);
e = d.length;
m = 3 * (e / 3);
g = (e - 1) / 3 + 1 << 2;
a = Array(g);
for (l = f = 0; f < m;) j = (d[f++] & 255) << 16 | (d[f++] & 255) << 8 | d[f++] & 255, a[l++] = b.charAt(j >> 18 & 63), a[l++] = b.charAt(j >> 12 & 63), a[l++] = b.charAt(j >> 6 & 63), a[l++] = b.charAt(j & 63);
f = e - m;
0 < f && (j = (d[m] &
255) << 10 | (2 == f ? (d[e - 1] & 255) << 2 : 0), a[g - 4] = b.charAt(j >> 12), a[g - 3] = b.charAt(j >> 6 & 63), a[g - 2] = 2 == f ? b.charAt(j & 63) : "=", a[g - 1] = "=");
return a.join("")
},
decode: function (c, h) {
var b, a, d, e, m, g, f, l, j, p, q, n;
b = h ? k.Base64.IAS : k.Base64.IA;
c.constructor == Array ? (d = c, m = !0) : (d = k.toByteArray(c), m = !1);
e = d.length;
g = 0;
for (f = e - 1; g < f && 0 > b[d[g]];) g++;
for (; 0 < f && 0 > b[d[f]];) f--;
l = "=" == d[f] ? "=" == d[f - 1] ? 2 : 1 : 0;
a = f - g + 1;
j = 76 < e ? ("\r" == d[76] ? a / 78 : 0) << 1 : 0;
e = (6 * (a - j) >> 3) - l;
a = Array(e);
q = p = 0;
for (eLen = 3 * (e / 3); p < eLen;) n = b[d[g++]] << 18 | b[d[g++]] <<
12 | b[d[g++]] << 6 | b[d[g++]], a[p++] = n >> 16 & 255, a[p++] = n >> 8 & 255, a[p++] = n & 255, 0 < j && 19 == ++q && (g += 2, q = 0);
if (p < e) {
for (j = n = 0; g <= f - l; j++) n |= b[d[g++]] << 18 - 6 * j;
for (b = 16; p < e; b -= 8) a[p++] = n >> b & 255
}
if (m) return a;
for (n = 0; n < a.length; n++) a[n] = String.fromCharCode(a[n]);
return a.join("")
}
};
k.Base64.init();
NBBY = 8;
MATCH_BITS = 6;
MATCH_MIN = 3;
MATCH_MAX = (1 << MATCH_BITS) + (MATCH_MIN - 1);
OFFSET_MASK = (1 << 16 - MATCH_BITS) - 1;
LEMPEL_SIZE = 256;
k.compress = function (c) {
var h = [],
b, a = 0,
d = 0,
e, m, g = 1 << NBBY - 1,
f, l, j = Array(LEMPEL_SIZE);
for (b = 0; b < LEMPEL_SIZE; b++) j[b] =
3435973836;
c = c.constructor == Array ? c : k.toByteArray(c);
for (b = c.length; a < b;) {
if ((g <<= 1) == 1 << NBBY) {
if (d >= b - 1 - 2 * NBBY) {
f = b;
for (d = a = 0; f; f--) h[d++] = c[a++];
break
}
g = 1;
m = d;
h[d++] = 0
}
if (a > b - MATCH_MAX) h[d++] = c[a++];
else if (e = (c[a] + 13 ^ c[a + 1] - 13 ^ c[a + 2]) & LEMPEL_SIZE - 1, l = a - j[e] & OFFSET_MASK, j[e] = a, e = a - l, 0 <= e && e != a && c[a] == c[e] && c[a + 1] == c[e + 1] && c[a + 2] == c[e + 2]) {
h[m] |= g;
for (f = MATCH_MIN; f < MATCH_MAX && c[a + f] == c[e + f]; f++);
h[d++] = f - MATCH_MIN << NBBY - MATCH_BITS | l >> NBBY;
h[d++] = l;
a += f
} else h[d++] = c[a++]
}
return h
};
k.decompress = function (c,
h) {
var b, a = [],
d, e = 0,
m = 0,
g, f, l = 1 << NBBY - 1,
j;
b = c.constructor == Array ? c : k.toByteArray(c);
for (d = b.length; e < d;) {
if ((l <<= 1) == 1 << NBBY) l = 1, f = b[e++];
if (f & l)
if (j = (b[e] >> NBBY - MATCH_BITS) + MATCH_MIN, g = (b[e] << NBBY | b[e + 1]) & OFFSET_MASK, e += 2, 0 <= (g = m - g))
for (; 0 <= --j;) a[m++] = a[g++];
else break;
else a[m++] = b[e++]
}
if (!("undefined" == typeof h ? 0 : h)) {
for (b = 0; b < m; b++) a[b] = String.fromCharCode(a[b]);
a = a.join("")
}
return a
}
})(fastcompressor);
And if memory serves... here's how you use it:
var compressed = fastcompressor.compress("0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789"); // data less than this length poses issues.
var decompressed = fastcompressor.decompress(compressed);
Rgds....Hoonto/Matt
Also, what I've posted is minified but beautified, and very slightly adapted for ease-of-use. Check the link in the update above for the original stuff.
这篇关于在javascript中压缩blob的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
