如何在TDB TripleStore中加载模型 [英] How I can load a model in TDB TripleStore

问题描述

我有一个问题：

我想在Jena TDB TripleStore上加载一个文件。
我的文件很大，大约80Mb，大约700000三元组RDF。当我尝试加载它时，执行停止工作或需要很长时间。

I would like to load a file on my Jena TDB TripleStore. My file is very big, about 80Mb and about 700000 triples RDF. When I try to load it, the execution stops working or takes a very long time.

我正在使用我在Web服务上运行的代码：

I'm using this code that I do run on a Web Service:

        String file = "C:\\file.nt";
        String directory;
        directory = "C:\\tdb";
        Dataset dataset = TDBFactory.createDataset(directory);

        Model model = ModelFactory.createDefaultModel();

        TDBLoader.loadModel(model, file );
        dataset.addNamedModel("http://nameFile", model); 

        return model;

有时我收到Java堆空间错误：

Sometimes I get an error of Java heap space:

Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.jena.riot.tokens.TokenizerText.parseToken(TokenizerText.java:170)
    at org.apache.jena.riot.tokens.TokenizerText.hasNext(TokenizerText.java:86)
    at org.apache.jena.atlas.iterator.PeekIterator.fill(PeekIterator.java:50)
    at org.apache.jena.atlas.iterator.PeekIterator.next(PeekIterator.java:92)
    at org.apache.jena.riot.lang.LangEngine.nextToken(LangEngine.java:99)
    at org.apache.jena.riot.lang.LangNTriples.parseOne(LangNTriples.java:67)
    at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:54)
    at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:42)
    at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTFactoryImpl$1.read(RDFParserRegistry.java:142)
    at org.apache.jena.riot.RDFDataMgr.process(RDFDataMgr.java:859)
    at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:255)
    at org.apache.jena.riot.RDFDataMgr.read(RDFDataMgr.java:241)
    at org.apache.jena.riot.adapters.RDFReaderRIOT_Web.read(RDFReaderRIOT_Web.java:96)
    at com.hp.hpl.jena.rdf.model.impl.ModelCom.read(ModelCom.java:241)
    at com.hp.hpl.jena.tdb.TDBLoader.loadAnything(TDBLoader.java:294)
    at com.hp.hpl.jena.tdb.TDBLoader.loadModel(TDBLoader.java:125)
    at com.hp.hpl.jena.tdb.TDBLoader.loadModel(TDBLoader.java:119)

如何在模型Jena中加载此文件并将其保存在TDB中？在此先感谢。

How I can load this file in a model Jena and save it in TDB? Thanks in advance.

推荐答案

您需要在statup上为你的JVM分配更多内存。当你太少时，进程将花费太多时间执行垃圾收集，并最终会失败。

You need to allocate more memory for your JVM at statup. When you have too little, the process will spend too much time performing garbage collection, and will ultimately fail.

例如，通过以下方式启动具有4 GB内存的JVM：

For example, start your JVM with 4 GB of memory by:

java -Xms4G -XmxG

如果您使用的是Eclipse等IDE，则可以更改运行配置，以便该应用程序还有额外的内存。

If you are in an IDE such as Eclipse, you can change your run configuration so that the application has additional memory as well.

除此之外，唯一突然出现的变化是您正在使用内存模型进行实际的加载操作，当你真的可以使用TDB支持的模型时。这可以帮助缓解您的内存问题，因为TDB动态地将其索引移动到磁盘。

Aside from that, the only change that jumps out at me is that you are using an in-memory model for the actual loading operation, when you can actually use a model backed by TDB instead. This can help to alleviate your memory problems because TDB dynamially moves its indexes to disk.

更改：

Dataset dataset = TDBFactory.createDataset(directory);
Model model = ModelFactory.createDefaultModel();
TDBLoader.loadModel(model, file );
dataset.addNamedModel("http://nameFile", model);

到此：

Dataset dataset = TDBFactory.createDataset(directory);
Model model = dataset.getNamedModel("http://nameFile");
TDBLoader.loadModel(model, file );

现在你的系统依赖于TDB能够做出关于何时将数据留在内存中以及何时使用将其刷新到磁盘。

Now your system depends on TDB's ability to make good decisions about when to leave data in memory and when to flush it to disk.

这篇关于如何在TDB TripleStore中加载模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！