斯坦福 CoreNLP 管道 coref:解析一些短字符串(很少提及)返回 indexoutofbounds 异常 [英] Stanford CoreNLP pipeline coref: parsing some short strings (with few mentions) returns indexoutofbounds exception
问题描述
背景:我正在将斯坦福 CoreNLP 库导入到我的 clojure 项目中.我使用的是 3.5.1 版，但最近直接跳到了 3.6.0 版，绕过了 3.5.2.作为此更新的一部分，因为我使用 dcoref 注释器获取共指信息，所以我需要进行一些小的修改，以便我的程序使用 coref 注释器.
在过去 (v3.5.1) 中，当我使用以下注释器创建管道时
In the past (v3.5.1), when I created a pipeline with the following annotators
"tokenize, ssplit, pos, lemma, ner, parse, depparse, dcoref, quote, entitymentions",
我可以毫无错误地解析如下语句:
I could parse a sentence such as the following without error:
我吃了面包".
如果我没记错的话，从生成的带注释的文档中提取共指链只会返回一个空值，或者可能是一个空数组.但这无关紧要，因为至少可以正确创建带注释的文档.
If I remember correctly, extracting the coreference chains from the resulting annotated document would just return an null value, or maybe an empty array. But that's inconsequential, because at least the annotated document would be created without error.
现在，当我使用以下注释器创建管道时:
Now, when I create a pipeline with the following annotators:
tokenize、ssplit、pos、lemma、ner、parse、depparse、mention、coref、quote、entitymention"、
然后我尝试解析同一个句子(或任何其他只有 1 或 0 个提及"的句子)我得到一个 indexoutofboundsexception 并带有以下跟踪:
and then I try to parse that same sentence (or any other sentences with only 1 or 0 "mentions") I get an indexoutofboundsexception with the following trace:
actual: java.lang.RuntimeException: Error annotating document with coref
at edu.stanford.nlp.scoref.StatisticalCorefSystem.annotate (StatisticalCorefSystem.java:79)
edu.stanford.nlp.scoref.StatisticalCorefSystem.annotate (StatisticalCorefSystem.java:62)
edu.stanford.nlp.pipeline.CorefAnnotator.annotate (CorefAnnotator.java:100)
edu.stanford.nlp.pipeline.AnnotationPipeline.annotate (AnnotationPipeline.java:68)
edu.stanford.nlp.pipeline.StanfordCoreNLP.annotate (StanfordCoreNLP.java:491)
nlp.core$parse_text.invoke (core.clj:199)
nlp.focus_scorer.process$lexchain_features.invoke (process.clj:63)
nlp.focus_scorer.process_test/fn (process_test.clj:49)
clojure.test$test_var$fn__7670.invoke (test.clj:704)
clojure.test$test_var.invoke (test.clj:704)
clojure.test$test_vars$fn__7692$fn__7697.invoke (test.clj:722)
clojure.test$default_fixture.invoke (test.clj:674)
clojure.test$test_vars$fn__7692.invoke (test.clj:722)
clojure.test$default_fixture.invoke (test.clj:674)
clojure.test$test_vars.invoke (test.clj:718)
clojure.test$test_all_vars.invoke (test.clj:728)
clojure.test$test_ns.invoke (test.clj:747)
clojure.core$map$fn__4553.invoke (core.clj:2624)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:49)
clojure.lang.Cons.next (Cons.java:39)
clojure.lang.RT.boundedLength (RT.java:1735)
clojure.lang.RestFn.applyTo (RestFn.java:130)
clojure.core$apply.invoke (core.clj:632)
clojure.test$run_tests.doInvoke (test.clj:762)
clojure.lang.RestFn.invoke (RestFn.java:408)
user$eval13163.invoke (form-init7737210093072696705.clj:1)
clojure.lang.Compiler.eval (Compiler.java:6782)
clojure.lang.Compiler.eval (Compiler.java:6745)
clojure.core$eval.invoke (core.clj:3081)
clojure.main$repl$read_eval_print__7099$fn__7102.invoke (main.clj:240)
clojure.main$repl$read_eval_print__7099.invoke (main.clj:240)
clojure.main$repl$fn__7108.invoke (main.clj:258)
clojure.main$repl.doInvoke (main.clj:258)
clojure.lang.RestFn.invoke (RestFn.java:1523)
clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__909.invoke (interruptible_eval.clj:58)
clojure.lang.AFn.applyToHelper (AFn.java:152)
clojure.lang.AFn.applyTo (AFn.java:144)
clojure.core$apply.invoke (core.clj:630)
clojure.core$with_bindings_STAR_.doInvoke (core.clj:1868)
clojure.lang.RestFn.invoke (RestFn.java:425)
clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke (interruptible_eval.clj:56)
clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__951$fn__954.invoke (interruptible_eval.clj:191)
clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__946.invoke (interruptible_eval.clj:159)
clojure.lang.AFn.run (AFn.java:22)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
java.lang.Thread.run (Thread.java:745)
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList$SubList.rangeCheck (ArrayList.java:1217)
java.util.ArrayList$SubList.get (ArrayList.java:1034)
edu.stanford.nlp.scoref.Clusterer$State.setClusters (Clusterer.java:349)
edu.stanford.nlp.scoref.Clusterer$State.<init> (Clusterer.java:322)
edu.stanford.nlp.scoref.Clusterer.getClusterMerges (Clusterer.java:58)
edu.stanford.nlp.scoref.ClusteringCorefSystem.runCoref (ClusteringCorefSystem.java:63)
edu.stanford.nlp.scoref.StatisticalCorefSystem.annotate (StatisticalCorefSystem.java:68)
edu.stanford.nlp.scoref.StatisticalCorefSystem.annotate (StatisticalCorefSystem.java:62)
edu.stanford.nlp.pipeline.CorefAnnotator.annotate (CorefAnnotator.java:100)
edu.stanford.nlp.pipeline.AnnotationPipeline.annotate (AnnotationPipeline.java:68)
edu.stanford.nlp.pipeline.StanfordCoreNLP.annotate (StanfordCoreNLP.java:491)
nlp.core$parse_text.invoke (core.clj:199)
nlp.focus_scorer.process$lexchain_features.invoke (process.clj:63)
nlp.focus_scorer.process_test/fn (process_test.clj:49)
clojure.test$test_var$fn__7670.invoke (test.clj:704)
clojure.test$test_var.invoke (test.clj:704)
clojure.test$test_vars$fn__7692$fn__7697.invoke (test.clj:722)
clojure.test$default_fixture.invoke (test.clj:674)
clojure.test$test_vars$fn__7692.invoke (test.clj:722)
clojure.test$default_fixture.invoke (test.clj:674)
clojure.test$test_vars.invoke (test.clj:718)
clojure.test$test_all_vars.invoke (test.clj:728)
clojure.test$test_ns.invoke (test.clj:747)
clojure.core$map$fn__4553.invoke (core.clj:2624)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:49)
clojure.lang.Cons.next (Cons.java:39)
clojure.lang.RT.boundedLength (RT.java:1735)
clojure.lang.RestFn.applyTo (RestFn.java:130)
clojure.core$apply.invoke (core.clj:632)
clojure.test$run_tests.doInvoke (test.clj:762)
clojure.lang.RestFn.invoke (RestFn.java:408)
user$eval13163.invoke (form-init7737210093072696705.clj:1)
clojure.lang.Compiler.eval (Compiler.java:6782)
clojure.lang.Compiler.eval (Compiler.java:6745)
clojure.core$eval.invoke (core.clj:3081)
clojure.main$repl$read_eval_print__7099$fn__7102.invoke (main.clj:240)
clojure.main$repl$read_eval_print__7099.invoke (main.clj:240)
clojure.main$repl$fn__7108.invoke (main.clj:258)
clojure.main$repl.doInvoke (main.clj:258)
clojure.lang.RestFn.invoke (RestFn.java:1523)
clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__909.invoke (interruptible_eval.clj:58)
clojure.lang.AFn.applyToHelper (AFn.java:152)
clojure.lang.AFn.applyTo (AFn.java:144)
clojure.core$apply.invoke (core.clj:630)
clojure.core$with_bindings_STAR_.doInvoke (core.clj:1868)
clojure.lang.RestFn.invoke (RestFn.java:425)
clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke (interruptible_eval.clj:56)
clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__951$fn__954.invoke (interruptible_eval.clj:191)
clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__946.invoke (interruptible_eval.clj:159)
clojure.lang.AFn.run (AFn.java:22)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
java.lang.Thread.run (Thread.java:745)
我可能做错了什么吗?我意识到我使用 clojure 而不是 java 的事实可能会导致一些问题，但我从来没有遇到过版本 3.5.1 的问题.似乎错误是从 edu.stanford.nlp.scoref.StatisticalCorefSystem.annotate 中的注释步骤抛出的，但我不确定我能对此做些什么(除了有两个管道对象，一个带有coref 注释器和一个没有，解析没有 coref 的句子，计算提及数，然后只有当我看到不止一个提及时才使用 coref 解析......这似乎有点太多了.)
Am I possibly doing something wrong? I realize that the fact that I'm using clojure instead of java might be causing some issue, but I've never had a problem with version 3.5.1. It would seem that the error is being thrown from the annotation step in edu.stanford.nlp.scoref.StatisticalCorefSystem.annotate, but I'm not sure what I can do about that (other than to have two pipeline objects, one with the coref annotator and one without, parse the sentence without coref, count the mentions, and then parse with coref only if I see more than one mention... which seems a little too much.)
推荐答案
3.6.0 对共指进行了重大更改.这个问题是斯坦福 CoreNLP 3.6.0 中的一个错误.如果你重新下载发行版，这个错误应该会在网站上的内容中得到修复.它也应该在即将发布的 Maven 版本中修复.
3.6.0 features major changes to coreference. This issue is a bug in Stanford CoreNLP 3.6.0. If you re-download the distribution this bug should be fixed in what's up on the site now. It should also be fixed in the up-coming Maven release.
这篇关于斯坦福 CoreNLP 管道 coref:解析一些短字符串(很少提及)返回 indexoutofbounds 异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
