使用流解密和解压缩以限制内存使用？ [英] Using streams to decrypt and unzip to limit memory usage?

问题描述

我有一个非常大的zip文件，2.5gb，这是加密的。我不能解密整个文件到内存和解压缩生产。所以我试图使用流来限制使用的内存量。

I have a very large zip file, 2.5gb, which is encrypted. I can't decrypt the entire file into memory and unzip there for production. So I'm trying to use streams to limit the amount of memory used.

我挂钩了以下操作（错误处理和流关闭左clear）：

I've hooked up the following to do it (error handling and stream closing left out for clarity):

SecretKeySpec keySpec = new SecretKeySpec(myKey "AES");
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");

FileInputStream fis = new FileInputStream(new File(pathToEncryptedFile));
CipherInputStream cis = new CipherInputStream(fis, cipher);

ZipInputStream zis = new ZipInputStream(new BufferedInputStream(cis));
ZipEntry ze = null;
while ((ze = zis.getNextEntry()) != null) {
    String filename = ze.getName();
    System.out.println("Found zip entry: " + filename);
}

这适用于大约50％的文件，压缩和加密相同的方式。我将在解压缩部分的while（）循环中获得异常：

This works for about 50% of my files, even though they're all zipped and encrypted the same way. The exception I'll get in the while() loop for the unzipping part:

java.util.zip.ZipException: unknown format (EXTSIG=f23f1090)
  at java.util.zip.ZipInputStream.readAndVerifyDataDescriptor(ZipInputStream.java:196)
  ...

如果我将整个文件解密到一个字节缓冲区并写入磁盘，那么对文件使用ZipInputStream，它适用于我所有的测试文件。

If I decrypt the entire file to a byte buffer and write it to disk, then use ZipInputStream on the file, it works for all my test files.

似乎加密文件末尾的额外填充在尝试使用流时会导致一些问题，但我认为PKCS5Padding规范会处理。

It seems like the extra padding at the end of the encrypted file is causing some problems when trying to use streams, but I thought the "PKCS5Padding" specification would take care of that.

感谢

推荐答案

在解密的档案中使用ZipInputStream 而不将其读入内存。如果失败，您的文件不能被读取，需要重新创建（可能是稍微不标准）。如果成功，写出解密流的结果（在传递给ZipInputStream之前）并检查二进制差异。

Use ZipInputStream on the decrypted file without reading it into memory. If that fails, your file cannot be read anyways, and needs to be recreated (could be that it's slightly non-standard). If it succeeds, write out the results of the decryption stream (before passing it to ZipInputStream) and check for binary differences.

这篇关于使用流解密和解压缩以限制内存使用？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！