解析二进制文件时出错...(主要是 PDF) [英] Error while parsing Binary Files... (mostly PDF)

问题描述

我正在尝试通过对二进制文件使用 ByteArrayInputStream 来使用 Apache Tika 解析 pdf 文件......并且开始出现一些 pdf 文件的错误，而对于一些它解析得很好.. 早些时候我能够解析相同的pdf 文件使用 Tika，但现在当我尝试使用 ByteArrayInputStream 时，我开始收到错误..我认为 ByteArray 存在一些问题这是我得到的错误..

I am trying to parse pdf file using Apache Tika by using ByteArrayInputStream for Binary files... And started getting error for some pdf file and for some it is parsing very well.. Earlier I was able to parse same pdf files using Tika, but now when I tried using ByteArrayInputStream, I started getting error..I think there is some problem with the ByteArray This is the Error I am getting..

org.apache.tika.exception.TikaException: Unexpected RuntimeException from org.apache.tika.parser.pdf.PDFParser@652489c0

这是我的代码...

if (page.isBinary()) {
   handleBinary(page, curURL);
}

<小时>

public int handleBinary(Page page, WebURL curURL) {
    try {
          binaryParser.parse(page.getBinaryData());
          page.setText(binaryParser.getText());
          handleMetaData(page, binaryParser.getMetaData());


          //System.out.println(" pdf url " +page.getWebURL().getURL());
          //System.out.println("Text" +page.getText());
    } catch (Exception e) {
          // TODO: handle exception
    }
          return PROCESS_OK;
}

<小时>

        public class BinaryParser {

            private String text;
            private Map<String, String> metaData;

            private Tika tika;

            public BinaryParser() {
                tika = new Tika();
            }

            public void parse(byte[] data) {
                InputStream is = null;
                try {
                    is = new ByteArrayInputStream(data);
                    text = null;
                    Metadata md = new Metadata();
                    metaData = new HashMap<String, String>();
                    text = tika.parseToString(is, md).trim();
                    processMetaData(md);
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    IOUtils.closeQuietly(is);
                }
            }

            public String getText() {
                return text;
            }

            public void setText(String text) {
                this.text = text;
            }


            private void processMetaData(Metadata md){
                if ((getMetaData() == null) || (!getMetaData().isEmpty())) {
                    setMetaData(new HashMap<String, String>());
                }
                for (String name : md.names()){
                    getMetaData().put(name.toLowerCase(), md.get(name));
                }
            }

            public Map<String, String> getMetaData() {
                return metaData;
            }

            public void setMetaData(Map<String, String> metaData) {
                this.metaData = metaData;
            }

        }

<小时>

    public class Page {

        private WebURL url;

        private String html;

        // Data for textual content
        private String text;

        private String title;

        private String keywords;
        private String authors;
        private String description;
        private String contentType;
        private String contentEncoding;

        private byte[] binaryData;

        private List<WebURL> urls;

        private ByteBuffer bBuf;

        private final static String defaultEncoding = Configurations
                .getStringProperty("crawler.default_encoding", "UTF-8");

        public boolean load(final InputStream in, final int totalsize,
                final boolean isBinary) {
            if (totalsize > 0) {
                this.bBuf = ByteBuffer.allocate(totalsize + 1024);
            } else {
                this.bBuf = ByteBuffer.allocate(PageFetcher.MAX_DOWNLOAD_SIZE);
            }
            final byte[] b = new byte[1024];
            int len;
            double finished = 0;
            try {
                while ((len = in.read(b)) != -1) {
                    if (finished + b.length > this.bBuf.capacity()) {
                        break;
                    }
                    this.bBuf.put(b, 0, len);
                    finished += len;
                }
            } catch (final BufferOverflowException boe) {
                System.out.println("Page size exceeds maximum allowed.");
                return false;
            } catch (final Exception e) {
                System.err.println(e.getMessage());
                return false;
            }

            this.bBuf.flip();
            if (isBinary) {
                binaryData = new byte[bBuf.limit()];
                bBuf.get(binaryData);
            } else {
                this.html = "";
                this.html += Charset.forName(defaultEncoding).decode(this.bBuf);
                this.bBuf.clear();
                if (this.html.length() == 0) {
                    return false;
                }
            }
            return true;
        }
    public boolean isBinary() {
        return binaryData != null;
    }

    public byte[] getBinaryData() {
        return binaryData;
    }

任何建议我做错了什么...!!

Any suggestions what wrong I am doing...!!

更新:-升级到 pdfbox 1.6.0 版本后，我开始收到一些 pdf 的这个错误...

UPDATED:- After upgrading to pdfbox 1.6.0 version, I started getting this error for some pdf...

Parsing Error, Skipping Object
java.io.IOException: expected='endstream' actual='' org.apache.pdfbox.io.PushBackInputStream@70dbdc4b
    at org.apache.pdfbox.pdfparser.BaseParser.parseCOSStream(BaseParser.java:439)
    at org.apache.pdfbox.pdfparser.PDFParser.parseObject(PDFParser.java:552)
    at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:184)
    at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1088)
    at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1053)

对于某些 pdf 这个错误...

And for some pdf this error...

 Did not found XRef object at specified startxref position 0
Invalid dictionary, found: '' but expected: '/'
 WARN [Crawler 2] Did not found XRef object at specified startxref position 0

推荐答案

这是 PDFBox 版本 1.4.0 的一个已知错误.只需更新到 PDFBox 1.5.0+.

This is a known bug of PDFBox version 1.4.0. Just update to PDFBox 1.5.0+.

查看此发行说明:

[PDFBOX-578] PDPageNode.getCount 中的 NPE NullPointerException

[PDFBOX-578] NPE NullPointerException in PDPageNode.getCount

还有这个 JIRA 票.

这篇关于解析二进制文件时出错...(主要是 PDF)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！