使用Stanford CoreNLP - Java堆空间 [英] Using Stanford CoreNLP - Java heap space

问题描述

我正在尝试使用Stanford CoreNLP管道的coreference模块，但最终我在Java中遇到了OutOfMemory错误。我已经增加了堆大小（通过Run-> Run Configurations-> Eclipse中的VM Arguments）并将它们设置为-Xmx3g -Xms1g。我甚至尝试过-Xmx12g -Xms4g，但这也没有用。我在64位计算机上使用带有Java 1.6的OS X 10.8.5上的Eclipse Juno。
有没有人知道我还能尝试什么？

I am trying to use the coreference module of the Stanford CoreNLP pipeline, but I end up getting an OutOfMemory error in Java. I already increased the heap size (via Run->Run Configurations->VM Arguments in Eclipse) and set them to -Xmx3g -Xms1g. I even tried -Xmx12g -Xms4g, but that didn't help either. I'm using Eclipse Juno on OS X 10.8.5 with Java 1.6 on a 64-bit machine. Does anyone have an idea what else I could try?

我正在使用网站上的示例代码（ http://nlp.stanford.edu/software/corenlp.shtml ）：

I'm using the example code from the website (http://nlp.stanford.edu/software/corenlp.shtml):

Properties props = new Properties();
props.put("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref");
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);

String text = "Stanford University is located in California. It is a great university";

Annotation document = new Annotation(text);

pipeline.annotate(document);

List<CoreMap> sentences = document.get(SentencesAnnotation.class);

for(CoreMap sentence: sentences) {
  for (CoreLabel token: sentence.get(TokensAnnotation.class)) {
    String word = token.get(TextAnnotation.class);
    String pos = token.get(PartOfSpeechAnnotation.class);
    String ne = token.get(NamedEntityTagAnnotation.class);       
  }

  Tree tree = sentence.get(TreeAnnotation.class);
  SemanticGraph dependencies = sentence.get(CollapsedCCProcessedDependenciesAnnotation.class);
}

Map<Integer, CorefChain> graph = document.get(CorefChainAnnotation.class);

我收到错误：

Adding annotator tokenize
Adding annotator ssplit
Adding annotator pos
Reading POS tagger model from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [0.9 sec].
Adding annotator lemma
Adding annotator ner
Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [3.1 sec].
Initializing JollyDayHoliday for sutime
Reading TokensRegex rules from edu/stanford/nlp/models/sutime/defs.sutime.txt
Reading TokensRegex rules from edu/stanford/nlp/models/sutime/english.sutime.txt
Jan 9, 2014 10:39:37 AM edu.stanford.nlp.ling.tokensregex.CoreMapExpressionExtractor appendRules
INFO: Ignoring inactive rule: temporal-composite-8:ranges
Reading TokensRegex rules from edu/stanford/nlp/models/sutime/english.holidays.sutime.txt
Adding annotator dcoref
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.String.substring(String.java:1939)
at java.lang.String.subSequence(String.java:1972)
at java.util.regex.Pattern.split(Pattern.java:1002)
at java.lang.String.split(String.java:2292)
at java.lang.String.split(String.java:2334)
at edu.stanford.nlp.dcoref.Dictionaries.loadGenderNumber(Dictionaries.java:382)
at edu.stanford.nlp.dcoref.Dictionaries.<init>(Dictionaries.java:553)
at edu.stanford.nlp.dcoref.Dictionaries.<init>(Dictionaries.java:463)
at edu.stanford.nlp.dcoref.SieveCoreferenceSystem.<init>(SieveCoreferenceSystem.java:282)
at edu.stanford.nlp.pipeline.DeterministicCorefAnnotator.<init>(DeterministicCorefAnnotator.java:52)
at edu.stanford.nlp.pipeline.StanfordCoreNLP$11.create(StanfordCoreNLP.java:775)
at edu.stanford.nlp.pipeline.AnnotatorPool.get(AnnotatorPool.java:81)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.construct(StanfordCoreNLP.java:260)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.<init>(StanfordCoreNLP.java:127)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.<init>(StanfordCoreNLP.java:123)
at extraction.BaselineApproach.main(BaselineApproach.java:88)

推荐答案

似乎问题不是斯坦福CoreNLP或Java，而是Eclipse。
这是我尝试过的：

通过打印 System.out.println（Runtime.getRuntime（）。maxMemory（））;检查参数是否实际传递到VM / em>

通过查看正在运行的进程以及它们正在使用的参数（ ps -al ）来检查Eclipse使用的参数

It seems the problem wasn't Stanford CoreNLP or Java, but Eclipse. Here's what I tried:

Checking if the arguments actually get passed through to the VM by printing System.out.println(Runtime.getRuntime().maxMemory());

Checking what arguments Eclipse uses by looking at what processes are running and what arguments they're using (ps -al)

然后证明Eclipse没有使用我指定的VM设置。

当我使用 Runtime.getRuntime（）。maxMemory（）检查时，输出为530186240而不是3203792896.

Eclipse使用的参数是错误的顺序，即 -Xms1g -Xmx3g -Xmx256m -Xmx512m 而不是 -Xmx256m -Xmx512m -Xms1g -Xmx3g 。由于只使用了最后一组参数，这就解释了为什么它不起作用。

It then turned out that Eclipse wasn't using the VM settings I specified.

When I checked with Runtime.getRuntime().maxMemory(), the output was 530186240 instead of 3203792896.

The arguments used by Eclipse were in the wrong order, i.e. -Xms1g -Xmx3g -Xmx256m -Xmx512m instead of -Xmx256m -Xmx512m -Xms1g -Xmx3g. Since only the last set of arguments are used, this explains why it's not working.

然后我尝试了以下方法来修复它：

不要覆盖特定应用程序的设置，但是而是覆盖默认设置（参见 http://help.eclipse.org/juno/index.jsp?topic=%2Forg.eclipse.pde.doc.user%2Fguide%2Ftools%2Flaunchers%2Farguments.htm ） 。

I then tried the following to fix it:

Don't override settings for a specific application, but rather override the default settings (see http://help.eclipse.org/juno/index.jsp?topic=%2Forg.eclipse.pde.doc.user%2Fguide%2Ftools%2Flaunchers%2Farguments.htm).

当这也不起作用时，我尝试：

手动编辑eclipse.ini文件（添加 -vmargs -Xms1g -Xmx3g ）

When that didn't work either, I tried to:

Manually edit the eclipse.ini file (add -vmargs -Xms1g -Xmx3g)

当这也不起作用时，我重新安装日食。现在一切正常：我可以设置默认设置并覆盖特定应用程序。

When that didn't work either, I re-installed Eclipse. Now everything works again: I can set default settings and override them for specific applications.

这篇关于使用Stanford CoreNLP - Java堆空间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！