Pdf Miner返回奇怪的字母/字符 [英] Pdf Miner returns weird letters/characters

问题描述

我在python 3中使用pdfminer，并且在从pdf中恢复的文本中出现了奇怪的字母.

I am using pdfminer with python 3 and I get weird letters in the text that is recovered from the pdf.

例如，我得到的是significant，而不是significant(请注意，字母f和I合并为一个).

For instance, I get significantinstead of significant (notice that the letters f and I are merged into one).

我不知道为什么会这样.这是我正在使用的代码.

I have no idea why this is happening. This is the code I am using.

from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage
from io import StringIO
from nltk.tokenize import sent_tokenize


def convert_pdf_to_txt(path):
    rsrcmgr = PDFResourceManager()
    retstr = StringIO()
    codec = 'utf-8'
    laparams = LAParams()
    device = TextConverter(rsrcmgr, retstr, codec=codec, laparams=laparams)
    fp = open(path, 'rb')
    interpreter = PDFPageInterpreter(rsrcmgr, device)
    password = ""
    maxpages = 0
    caching = True
    pagenos=set()

    for page in PDFPage.get_pages(fp, pagenos, maxpages=maxpages, password=password,caching=caching, check_extractable=True):
        interpreter.process_page(page)

    text = retstr.getvalue()

    sentences = sent_tokenize(text)

    for s in sentences:
        print(s)
        print("\n\n")

到目前为止，我唯一的猜测是它可能与编码有关，但似乎

My only guess so far is that it may have to do with the encoding, but it seems that there is no way to retrieve the encoding of a pdf

推荐答案

PDFminer正常工作.有问题的字符是Unicode字符U + FB01， fi连字.

PDFminer is working correctly. The character in question is the Unicode character U+FB01, the fi ligature.

在代码中添加一行以fi替换fi:

Add a line to replace fi with fi to your code:

for s in sentences:
    s = s.replace ('fi', 'fi')
    print (s)

还有另一种非常常见的-纯印刷(*)-用Unicode定义的连字:U + FB02，fl连字；一样对待:

There is one other very common – and purely typographic(*) – ligature defined in Unicode: U+FB02, the fl ligature; treat this the same:

    s = s.replace ('fl', 'fl')

以及字母表示，您最好也包括在内.

and a couple of others in the Alphabetic Presentation block, which you might as well include too.

(*)不犯错误以更改 œ 到oe.这些不是 纯印刷连字"，而是有效的字符.

(*) Do not make the mistake to change æ to ae and œ to oe. These are not 'purely typographic ligatures' but valid characters on their own.

这篇关于Pdf Miner返回奇怪的字母/字符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！