在java中压缩和解压大型数据？ [英] Compressing and decompressing large size data in java?

问题描述

我需要压缩/解压缩包含在一个文件夹中的不同类型的文件，该文件夹的大小可能超过10-11 GB。
我使用下面的代码，但是这需要很长时间来压缩数据。

  BufferedReader in = new BufferedReader的FileReader（ d：/ziptest/expansion1.MPQ））; 
 BufferedOutputStream out = new BufferedOutputStream（
 new GZIPOutputStream（new FileOutputStream（test.gz）））; 
 
 int c; （（c = in.read（））！= -1）
 out.write（c）; 
 in.close（）; 
 out.close（）; 
  

请在java中建议我使用一些快速压缩和解压缩库，我还想将大文件不同的部分，例如每个100MB的块。

解决方案

与这些是会得到损坏。

反而我建议你使用FileInputStream。 InputStream in = new FileInputStream（D：/ ziptest /文件名 expansion1.MPQ）;
OutputStream out = new GZIPOutputStream（
new BufferedOutputStream（new FileOutputStream（test.gz）））;

byte [] bytes = new byte [32 * 1024];
int len; （（len = in.read（bytes））> 0）
out.write（bytes，0，len）;

in.close（）;
out.close（）;


$ b $ p
$ b $ p 由于您读取大块字节，因此不会对BufferedInput / OuptuStream更有效，复制。在GZIPOutputStream之后有一个BufferedOutptuStream，因为你不能控制它产生的数据的大小。

BTW：如果你只是用Java读这个，你可以使用DeflatorOutputStream，稍微快一点，但只支持Java AFAIK。

I need to compress/decompress different types of files that are contained in a Folder the size of that folder might be more than 10-11 GB. I used following code but this is taking long time to compress the data.

BufferedReader in = new BufferedReader(new FileReader("D:/ziptest/expansion1.MPQ"));
BufferedOutputStream out = new BufferedOutputStream(
    new GZIPOutputStream(new FileOutputStream("test.gz")));

int c;
while ((c = in.read()) != -1)
  out.write(c);
in.close();
out.close();

Please suggest me some fast compressing and decompressing library in java, i also want to split the large file in different parts such as in a chunk of 100MB each.

解决方案

Reader/Writer is only for Text and if you try to read binary with these is will get corrupted.

Instead I suggest you use FileInputStream. The fastest way to copy the data is to use your own buffer.

InputStream in = new FileInputStream("D:/ziptest/expansion1.MPQ");
OutputStream out = new GZIPOutputStream(
            new BufferedOutputStream(new FileOutputStream("test.gz")));

byte[] bytes = new byte[32*1024];
int len;
while((len = in.read(bytes)) > 0)
   out.write(bytes, 0, len);

in.close();
out.close();

Since you reading large chunks of bytes, it is more efficient not to BufferedInput/OuptuStream as this removes one copy. There is a BufferedOutptuStream after the GZIPOutputStream as you cannot control the size of data it produces.

BTW: If you are only reading this with Java, you can use DeflatorOutputStream, its slightly faster and smaller, but only supported by Java AFAIK.

这篇关于在java中压缩和解压大型数据？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！