Stanford CoreNLP Annotators线程安全吗？ [英] Stanford CoreNLP Annotators Thread safe?

问题描述

Stanford CoreNLP网站

the website of Stanford CoreNLP

http://nlp.stanford.edu/software/corenlp.shtml

列出了几十个像魅力一样工作的注释器。我想使用Annotators的实例来处理多个线程的常见任务（词形还原，标记，解析）。例如，将大量（GB文本）的处理拆分为线程或提供Web服务。

lists dozens of Annotators which work like a charm. I would like to use instances of the Annotators for the common tasks (lemmatization, tagging, parsing) by multiple threads. For example to split up the processing of a massively large (GBs of Text) into threads or to provide web services.

过去有过一些关于LocalThreads的讨论根据我的理解，每个线程使用一个Annotator实例（从而避免了线程安全方面的问题）。这是一个选项，但是这样所有模型文件和资源也必须加载n次。

There has been some discussion in the past referring to LocalThreads which, by my understanding, use one instance of an Annotator per Thread (thus avoiding problems regarding thread-safety). This is an option but that way all model files and resources have to be loaded n times as well.

Annotators（或其中一些）是否可以使用线程安全？我在讨论，文档或常见问题中找不到任何结论性/官方性。

Are the Annotators (or some of them) thread-safe to use? I couldn't find anything conclusive/official in the discussions, docs or faqs.

推荐答案

是的，注释器的目的是线程安全的。您可以创建一个新的AnnotationPipeline（例如，一个新的StanfordCoreNLP对象），然后许多线程可以将注释传递到此管道中，而无需为每个线程重新加载模型。

Yes, the annotators are intended to be thread-safe. You can create a new AnnotationPipeline (e.g., a new StanfordCoreNLP object), and then many threads can pass annotations into this pipeline without reloading the model for each thread.

这篇关于Stanford CoreNLP Annotators线程安全吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！