签署PDF - 内存消耗 [英] Signing PDF - memory consumption

问题描述

我尝试了一些基于iText v1或v2的数字PDF签名工具，发现似乎整个PDF被加载到内存中（对于60M PDF进程可能需要300-400MB的内存）。

最近的iText版本可以在不加载内存的情况下签署PDF吗？

更新

我用itextpdf测试了Bruno的例子5.5.6

• PdfReader构造函数没关系 - 它可以是（src）或（src） ，null，true）或
  （src，null，false） - 结果相同。


• 重要的是createSignature中的新文件（tmp）。

但内存消耗仍然很大。我试图签署100M文件（它带有嵌入式附件的PDF），峰值内存大约是325M。当然，它比没有临时文件的540M更好，但还不够好（（<。

32K文件最大内存为65M（那是JVM和java代码本身，我猜测）

内存是用 / usr / bin / time -v java ....

我使用 -Xmx100m 限制了Java内存，但是因为内存不足而崩溃：

线程main中的异常java.lang.OutOfMemoryError：Java堆空间

at com.itextpdf.text。 pdf.PdfReader.getStreamBytesRaw（PdfReader.java:2576）
at com.itextpdf.text.pdf.PdfReader.getStreamBytesRaw(pdfReader.java:2615）
at com.itextpdf.text.pdf.PRStream。 toPdf（PRStream.java:230）
at com.itextpdf.text.pdf.PdfIndirectObject.writeTo（PdfIndirectObject.java:158）
at com.itextpdf.text.pdf.PdfWriter $ PdfBody.write（ PdfWriter.java:420）
at com.itextpdf.text.pdf.PdfWriter $ PdfBody.add（PdfWriter.java:398）
at com.itextpdf.tex t.pd​​f.PdfWriter.addToBody（PdfWriter.java:887）
at com.itextpdf.text.pdf.PdfStamperImp.close（PdfStamperImp.java:412）
at com.itextpdf.text.pdf。 PdfStamperImp.close（PdfStamperImp.java:386）
at com.itextpdf.text.pdf.PdfSignatureAppearance.preClose（PdfSignatureAppearance.java:1316）
at com.itextpdf.text.pdf.security.MakeSignature。 signDetached（MakeSignature.java:140）

代码是：

  public static byte [] getStreamBytesRaw（final PRStream stream，final RandomAccessFileOrArray file）抛出IOException {
 PdfReader reader = stream.getReader（）; 
 byte b []; 
 if（stream.getOffset（）< 0）
 b = stream.getBytes（）; 
 else {
 ----> b =新字节[stream.getLength（）]; 
 file.readFully（b）; 
  

我在调试器中看到流类型是EmbeddedFile，长度是100M - 所以正在读取整个嵌入文件进入内存。

更新 - 创建大PDF

分享100M文件很难）），但这里是创建序列：

1. 运行 dd if = / dev / urandom of = file.bin bs = 1048000 count = 100


2. 转到 http： //blog.didierstevens.com/programs/pdf-tools/ 并参加 http ：//didierstevens.com/files/software/make-pdf_V0_1_6.zip


3. 解压缩并运行 python make-pdf-embedded.py文件。 bin file.pdf

在这里你是）

我应该注意使用 / dev / urandom 很重要。 / dev / zero 创建只有100K大小的压缩PDF。

无论如何，如果需要获取我的文件，我已经创建了50M文件服务器 - http://50mpdf.tk/50m.pdf

解决方案

签署PDF时，iText使用相关的内存量

• 阅读除非在部分模式下使用 PdfReader ;


• 在内存中创建签名文件，否则将整个未签名的PDF存入内存除非使用 PdfStamper 配置为使用临时文件;并且


• 在将无符号数据复制到要签名的文件时将整个单独的PDF对象（例如包含嵌入文件的流）读入内存，除非使用 PdfStamper 在附加模式。

例如签署样本提供的50 MB文件需要

• 关于 -Xmx240m 如果既不使用附加模式，也不使用临时文件，也不使用部分模式;


• 关于 -Xmx81m 如果使用临时文件但不附加模式，部分模式没有区别;


• 关于 -Xmx7m 如果使用追加模式和临时文件，则部分模式没有区别。

部分模式在后来的情况下没有区别的原因是，即使在非部分模式下， PdfReader 在初始化期间似乎不读取流内容。由于示例文件主要由单个大流的内容组成，因此在初始化期间读取或未读取的少数对象没有区别，特别是在部分模式下， PdfReader 读取并保留一些反映全局文档结构的内存中的对象，例如页面树。

您可以在此处找到我的测试例程： CreateSignature.java 。我使用iText 5.5.7-SNAPSHOT在64位MS Windows Java 8上运行它（在这种情况下，它不应该与5.5.6版本不同）。

因此，对于内存友好的签名使用@ Bruno代码的这个变体：

  //创建读卡器和压模
 PdfReader reader = new PdfReader（filepath，null，true）; 
 FileOutputStream os = new FileOutputStream（dest）; 
 PdfStamper压模= 
 PdfStamper.createSignature（reader，os，'\ 0'，新文件（tmp），true）; 
 //创建外观
 PdfSignatureAppearance appearance = stamper.getSignatureAppearance（）; 
 appearance.setReason（reason）; 
 appearance.setLocation（location）; 
 appearance.setVisibleSignature（new Rectangle（36,748,144,780），1，sig）; 
 //创建签名
 ExternalSignature pks = new PrivateKeySignature（pk，digestAlgorithm，provider）; 
 ExternalDigest digest = new BouncyCastleDigest（）; 
 MakeSignature.signDetached（外观，摘要，pks，链，
 null，null，null，0，subfilter）; 
  

I tried some utilities for digital PDF signing based on iText v1 or v2 and found out that it seems whole PDF is loaded into memory (for 60M PDF process can take up to 300-400MB of memory).

Can recent iText versions sign PDF without load it into memory?

Updates

I tested Bruno's example with itextpdf 5.5.6

• PdfReader constructor doesn't matter - it can be (src) or (src, null, true), or (src, null, false) - result the same.
• what matters is new File(tmp) in createSignature.

But memory consumption is still to big. I tried to sign 100M file (it's PDF with embedded attachment), peak memory is about 325M. Sure, it's better than 540M without temporary file, but not good enough (((.

With 32K file max. memory was 65M (that's JVM and java code itself, I guess)

Memory was measured with /usr/bin/time -v java ....

I limited Java memory with -Xmx100m, but it crashed with out of memory:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

at com.itextpdf.text.pdf.PdfReader.getStreamBytesRaw(PdfReader.java:2576) at com.itextpdf.text.pdf.PdfReader.getStreamBytesRaw(PdfReader.java:2615) at com.itextpdf.text.pdf.PRStream.toPdf(PRStream.java:230) at com.itextpdf.text.pdf.PdfIndirectObject.writeTo(PdfIndirectObject.java:158) at com.itextpdf.text.pdf.PdfWriter$PdfBody.write(PdfWriter.java:420) at com.itextpdf.text.pdf.PdfWriter$PdfBody.add(PdfWriter.java:398) at com.itextpdf.text.pdf.PdfWriter.addToBody(PdfWriter.java:887) at com.itextpdf.text.pdf.PdfStamperImp.close(PdfStamperImp.java:412) at com.itextpdf.text.pdf.PdfStamperImp.close(PdfStamperImp.java:386) at com.itextpdf.text.pdf.PdfSignatureAppearance.preClose(PdfSignatureAppearance.java:1316) at com.itextpdf.text.pdf.security.MakeSignature.signDetached(MakeSignature.java:140)

Code is:

public static byte[] getStreamBytesRaw(final PRStream stream, final RandomAccessFileOrArray file) throws IOException {
        PdfReader reader = stream.getReader();
        byte b[];
        if (stream.getOffset() < 0)
            b = stream.getBytes();
        else {
      ----> b = new byte[stream.getLength()];
            file.readFully(b);

I see in debugger that stream type is EmbeddedFile and length is 100M - so whole embedded file is being read into memory.

Update - create big PDF

It's difficult to share 100M file )), but here is create sequence:

1. Run dd if=/dev/urandom of=file.bin bs=1048000 count=100
2. Go to http://blog.didierstevens.com/programs/pdf-tools/ and take http://didierstevens.com/files/software/make-pdf_V0_1_6.zip
3. Unzip and run python make-pdf-embedded.py file.bin file.pdf

Here you are )

I should note that it's important to use /dev/urandom. /dev/zero creates compressed PDF with only 100K size.

Anyway, if it's necessary to obtain my file I've created 50M file on server - http://50mpdf.tk/50m.pdf

解决方案

While signing a PDF, iText uses relevant amounts of memory

• reading the whole unsigned PDF into memory unless using a PdfReader in partial mode;
• creating the signed file in memory unless using a PdfStamper configured to use a temporary file; and
• reading whole individual PDF objects (e.g. streams containing embedded files) into memory when copying the unsigned data to the to-be-signed file unless using a PdfStamper in append mode.

E.g. signing the sample 50 MB file supplied by the OP requires

• about -Xmx240m if using neither append mode, nor a temporary file, nor partial mode;
• about -Xmx81m if using a temporary file but not append mode, partial mode makes no difference;
• about -Xmx7m if using append mode and a temporary file, partial mode makes no difference.

The reason why partial mode makes no difference in the later cases, is that even in non-partial-mode the PdfReader does not seems to read stream contents during initialization. As the sample file consists mostly of the contents of a single big stream, the few objects read or not read during initialization don't make a difference, especially as even in partial mode the PdfReader reads and keeps some objects in memory which reflect the global document structure, e.g. the page tree.

You can find my test routines here: CreateSignature.java. I ran it on a 64bit MS Windows Java 8 using iText 5.5.7-SNAPSHOT (which should not differ from the 5.5.6 release in this context).

Thus, for memory-friendly signing use this variant of @Bruno's code:

// Creating the reader and the stamper
PdfReader reader = new PdfReader(filepath, null, true);
FileOutputStream os = new FileOutputStream(dest);
PdfStamper stamper =
    PdfStamper.createSignature(reader, os, '\0', new File(tmp), true);
// Creating the appearance
PdfSignatureAppearance appearance = stamper.getSignatureAppearance();
appearance.setReason(reason);
appearance.setLocation(location);
appearance.setVisibleSignature(new Rectangle(36, 748, 144, 780), 1, "sig");
// Creating the signature
ExternalSignature pks = new PrivateKeySignature(pk, digestAlgorithm, provider);
ExternalDigest digest = new BouncyCastleDigest();
MakeSignature.signDetached(appearance, digest, pks, chain,
    null, null, null, 0, subfilter);

这篇关于签署PDF - 内存消耗的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！