将语言配置文件添加到Apache Tika [英] Adding language profile to Apache Tika
问题描述
请能够做到这一点的任何人解释一下如何做到这一点:-)
Could please anybody who managed to do that explain how to do that :-)
我需要获取需要添加的语言的n-gram文件吗?
Do I need to get n-gram files for the language I need to add ?
是否需要创建tika.language.override.properties，添加其他一些语言代码并在classPath上添加lang-code.ngp n-gram文件?在那种情况下，如果只是这个问题，我该从哪里得到呢?为什么Tika不支持更多的语言?
Is it a matter of creating tika.language.override.properties, add some other lang codes and add lang-code.ngp n-gram file on the classPath ? In that case, where do I get it and why Tika doesn't support more languages, if it is just a matter of this ?
当前支持这些语言以进行语言检测
There are currently these languages supported for language detection
da,de,et,el,en,es,fi,fr,hu,is,it,lt,nl,no,pl,pt,ru,sv,th
和tika使用传统的n-gram表示法
and tika uses traditional n-gram notation
er_ 132232
_de 103517
en_ 82666
et_ 80661
for 65286
_fo 57945
de_ 51382
der 44049
at_ 41915
det 41381
_og 40344
_at 39482
ing 38707
den 36795
og_ 36577
_me 34924
nde 34528
该语言检测应用程序当前支持这些语言，但是具有一些不同的n-克文件
This lang detection application currently supports these languages, but has kinda different n-gram files
af bg cs de en fa fr he hr id ja ko ml ne no pl ro sk sq sw te tl uk vi zh-tw ar bn da el es fi gu hi hu it kn mk mr nl pa pt ru so sv ta th tr ur zh-cn
JSON格式
{"freq":{"D":9246,"E":2445,"F":2510,"G":3299,"A":6930,"B":3706,"C":2451,"L":2519,"M":3951,"N":3334,"O":2514,"H" ....
推荐答案
从 TIKA-490 ，应该可以添加新的语言配置文件. TIKA-546 似乎表明它还不像以前那么容易，与此同时，您需要从Nutch的NGramProfile工具开始并调整输出.
It looks like as of TIKA-490, it should be possible to add new language profiles. TIKA-546 seems to indicate it isn't yet as easy as it might be, and in the mean time you'll need to start with Nutch's NGramProfile tool and tweak the output.
我建议您尝试使用Nutch工具生成文件，然后查看TIKA-490上的注释以获取有关如何使用它们的详细信息.
I'd suggest you try using the Nutch tool to generate the files, then look at the comments on TIKA-490 for details on how to use them.
这篇关于将语言配置文件添加到Apache Tika的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
