不断增长的 eXist 数据库 [英] Growing eXist database

查看：55
本文介绍了不断增长的 eXist 数据库的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我正在使用 Orbeon Forms 和 eXist 来存储填写好的表单.我还有下一个问题，eXist 每天增长超过 500 MB.如果我做整个数据库的保留副本，然后将删除所有数据库文件，并恢复保留副本，那么数据库只有 1 GB 大，但几天后它变成 4 GB，并且增长，增长和增长......

一开始我使用的是嵌入式 eXist 1.2.6，我认为它是不稳定的版本，所以我将所有数据移到了外部 eXist 1.4.1，但它仍然在新版本中继续增长.

这是来自exist/WEB-INF的conf.xml

<db-connection cacheSize="200M" collectionCache="200M" database="native" files="/opt/alfresco-3.4.c/tomcat/work/exist/data/" pageSize="4096"><recovery enabled="yes" group-commit="no" journal-dir="/opt/alfresco-3.4.c/tomcat/work/exist/recovery-data/" size="100M" sync-on-commit="yes" force-restart="yes" 一致性检查="yes"/><watchdog output-size-limit="10000" query-timeout="180000"/></db-connection><indexer caseSensitive="yes" index-depth="5" preserve-whitespace-mixed-content="yes" stemming="no" suppress-whitespace="none" tokenizer="org.exist.storage.analysis.SimpleTokenizer" track-term-freq="yes"><模块><module id="ngram-index" file="ngram.dbx" n="3" class="org.exist.indexing.ngram.NGramIndex"/><module id="lucene-index" buffer="32" class="org.exist.indexing.lucene.LuceneIndex"/></模块><停用词文件=停用词"/><索引><全文属性=假"默认=无"><exclude path="/auth"/></全文></index></索引器><调度器></调度器><serializer add-exist-id="none" compress-output="no" enable-xinclude="yes" enable-xsl="no" indent="yes" match-tagging-attributes="no" match-tagging-元素=否"></序列化器><transformer class="org.apache.xalan.processor.TransformerFactoryImpl" caching="yes"/><验证模式=否"><实体解析器><catalog uri="${WEBAPP_HOME}/WEB-INF/catalog.xml"/></实体解析器></验证><xquery enable-java-binding="no" disable-deprecated-functions="no" enable-query-rewriting="yes" backwardCompatible="no" raise-error-on-failed-retrieval="no"><内置模块><module uri="http://exist-db.org/xquery/lucene" class="org.exist.xquery.modules.lucene.LuceneModule"/><module uri="http://exist-db.org/xquery/ngram" class="org.exist.xquery.modules.ngram.NGramModule"/><module uri="http://exist-db.org/xquery/examples" class="org.exist.xquery.modules.example.ExampleModule"/><module uri="http://exist-db.org/xquery/request" class="org.exist.xquery.functions.request.RequestModule"/><module uri="http://exist-db.org/xquery/response" class="org.exist.xquery.functions.response.ResponseModule"/><module uri="http://exist-db.org/xquery/session" class="org.exist.xquery.functions.session.SessionModule"/><module uri="http://exist-db.org/xquery/system" class="org.exist.xquery.functions.system.SystemModule"/><module uri="http://exist-db.org/xquery/text" class="org.exist.xquery.functions.text.TextModule"/><module uri="http://exist-db.org/xquery/transform" class="org.exist.xquery.functions.transform.TransformModule"/><module uri="http://exist-db.org/xquery/util" class="org.exist.xquery.functions.util.UtilModule"/><module uri="http://exist-db.org/xquery/validation" class="org.exist.xquery.functions.validation.ValidationModule"/><module uri="http://exist-db.org/xquery/xmldb" class="org.exist.xquery.functions.xmldb.XMLDBModule"/><module uri="http://expath.org/ns/http-client" class="org.expath.exist.HttpClientModule"/><module uri="http://www.expath.org/mod/http-client" src="resource:org/expath/www/mod/http-client/http-client.xqm"/><module uri="http://exist-db.org/xquery/httpclient" class="org.exist.xquery.modules.httpclient.HTTPClientModule"/><module uri="http://exist-db.org/xquery/kwic" src="resource:org/exist/xquery/lib/kwic.xql"/><module uri="http://www.json.org" src="resource:org/exist/xquery/lib/json.xq"/><module uri="http://exist-db.org/xquery/sequences" src="resource:org/exist/xquery/lib/sequences.xq"/><module uri="http://exist-db.org/versioning" src="resource:org/exist/versioning/xquery/versioning.xqm"/><module uri="http://exist-db.org/xquery/testing" src="resource:org/exist/xquery/lib/test.xq"/><module uri="http://xproc.net/xproc" src="resource:net/xproc/xprocxq/src/xquery/xproc.xqm"/><module uri="http://xproc.net/xproc/const" src="resource:net/xproc/xprocxq/src/xquery/const.xqm"/><module uri="http://xproc.net/xproc/ext" src="resource:net/xproc/xprocxq/src/xquery/ext.xqm"/><module uri="http://xproc.net/xproc/functions" src="resource:net/xproc/xprocxq/src/xquery/functions.xqm"/><module uri="http://xproc.net/xproc/naming" src="resource:net/xproc/xprocxq/src/xquery/naming.xqm"/><module uri="http://xproc.net/xproc/opt" src="resource:net/xproc/xprocxq/src/xquery/opt.xqm"/><module uri="http://xproc.net/xproc/std" src="resource:net/xproc/xprocxq/src/xquery/std.xqm"/><module uri="http://xproc.net/xproc/util" src="resource:net/xproc/xprocxq/src/xquery/util.xqm"/><module uri="http://exist-db.org/xquery/file" class="org.exist.xquery.modules.file.FileModule"/><module uri="http://exist-db.org/xquery/xqdoc" class="org.exist.xqdoc.xquery.XQDocModule"/></内置模块></xquery><xupdate allowed-fragmentation="5" enable-consistency-checks="no"/></存在>

这是一个常见问题吗?或者我的 eXist 配置有误?

感谢您的帮助.

解决方案

感谢 @joewiz 和 @avernet

现在我可以回答我自己的问题了.主要问题是我使用的是 orbeon 3.8，它使用 eXist DB 在那里存储缓存.更新到 3.9 版本会有所帮助，因为 3.9 使用 Ehcahe.

作为临时解决方法，我将 orbeon 3.8 配置为将数据存储到外部 eXist，以便在 properties-local.xml 我添加

<property as="xs:anyURI" name="oxf.fr.persistence.service.exist.uri" value="http://127.0.0.1:8080/exist/rest/db"/>

属性如此处，并嵌入 eXist 以存储应设置下一个属性的缓存

<property as="xs:string" name="oxf.xforms.store.application.collection" value="/db/orbeon/xforms/cache/"/>

所以当 servlet 容器停止时，我可以安全地删除嵌入的 eXist 数据文件.

I'm using Orbeon Forms, and eXist to store filled forms there. And I have next problem, eXist is growing more than 500 MB per day. If I will made reserve copy of whole database, then will delete all DB files, and restore reserved copy, then DB is only 1 GB large, but after few days it becomes 4 GB, and grows, grows and grows...

First I was using embeded eXist 1.2.6, and I thought that it's unstable version, so I moved all data to external eXist 1.4.1, but still it continue to grow in new version.

Here's conf.xml from exist/WEB-INF

<?xml version="1.0" encoding="UTF-8"?>
<exist xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="schema/conf.xsd">
    <cluster dbaPassword="" dbaUser="admin" exclude="/db/system,/db/system/config" 
    journalDir="data/journal" 
    protocol="UDP(mcast_addr=228.1.2.3;mcast_port=45566;ip_ttl=8;ip_mcast=true;mcast_send_buf_size=800000;mcast_recv_buf_size=150000;ucast_send_buf_size=800000;ucast_recv_buf_size=150000;loopback=true):PING(timeout=2000;num_initial_members=3;up_thread=true;down_thread=true):MERGE2(min_interval=10000;max_interval=20000):FD(shun=true;up_thread=true;down_thread=true;timeout=2500;max_tries=5):VERIFY_SUSPECT(timeout=3000;num_msgs=3;up_thread=true;down_thread=true):pbcast.NAKACK(gc_lag=50;retransmit_timeout=300,600,1200,2400,4800;max_xmit_size=8192;up_thread=true;down_thread=true):UNICAST(timeout=300,600,1200,2400,4800;window_size=100;min_threshold=10;down_thread=true):pbcast.STABLE(desired_avg_gossip=20000;up_thread=true;down_thread=true):FRAG(frag_size=8192;down_thread=true;up_thread=true):pbcast.GMS(join_timeout=5000;join_retry_timeout=2000;shun=true;print_local_addr=true)"/>


    <db-connection cacheSize="200M" collectionCache="200M" database="native" files="/opt/alfresco-3.4.c/tomcat/work/exist/data/" pageSize="4096">

        <recovery enabled="yes" group-commit="no" journal-dir="/opt/alfresco-3.4.c/tomcat/work/exist/recovery-data/" size="100M" sync-on-commit="yes" force-restart="yes" consistency-check="yes"/>

        <watchdog output-size-limit="10000" query-timeout="180000"/>

    </db-connection>

    <indexer caseSensitive="yes" index-depth="5" preserve-whitespace-mixed-content="yes" stemming="no" suppress-whitespace="none" tokenizer="org.exist.storage.analysis.SimpleTokenizer" track-term-freq="yes">

            <modules>
            <module id="ngram-index" file="ngram.dbx" n="3" class="org.exist.indexing.ngram.NGramIndex"/>

            <module id="lucene-index" buffer="32" class="org.exist.indexing.lucene.LuceneIndex"/>
        </modules>

        <stopwords file="stopword"/>

        <index>
            <fulltext attributes="false" default="none">
                <exclude path="/auth"/>
            </fulltext>
        </index>
    </indexer>

    <scheduler>

    </scheduler>
    <serializer add-exist-id="none" compress-output="no" enable-xinclude="yes" enable-xsl="no" indent="yes" match-tagging-attributes="no" match-tagging-elements="no">
    </serializer>

    <transformer class="org.apache.xalan.processor.TransformerFactoryImpl" caching="yes"/>

    <validation mode="no">
        <entity-resolver>
            <catalog uri="${WEBAPP_HOME}/WEB-INF/catalog.xml"/>
        </entity-resolver>
    </validation>

    <xquery enable-java-binding="no" disable-deprecated-functions="no" enable-query-rewriting="yes" backwardCompatible="no" raise-error-on-failed-retrieval="no">
        <builtin-modules>
            <module uri="http://exist-db.org/xquery/lucene" class="org.exist.xquery.modules.lucene.LuceneModule"/>          
            <module uri="http://exist-db.org/xquery/ngram" class="org.exist.xquery.modules.ngram.NGramModule"/>

            <module uri="http://exist-db.org/xquery/examples" class="org.exist.xquery.modules.example.ExampleModule"/>           
            <module uri="http://exist-db.org/xquery/request" class="org.exist.xquery.functions.request.RequestModule"/>            
            <module uri="http://exist-db.org/xquery/response" class="org.exist.xquery.functions.response.ResponseModule"/>            
            <module uri="http://exist-db.org/xquery/session" class="org.exist.xquery.functions.session.SessionModule"/>            
            <module uri="http://exist-db.org/xquery/system" class="org.exist.xquery.functions.system.SystemModule"/>
            <module uri="http://exist-db.org/xquery/text" class="org.exist.xquery.functions.text.TextModule"/>           
            <module uri="http://exist-db.org/xquery/transform" class="org.exist.xquery.functions.transform.TransformModule"/>            
            <module uri="http://exist-db.org/xquery/util" class="org.exist.xquery.functions.util.UtilModule"/>            
            <module uri="http://exist-db.org/xquery/validation" class="org.exist.xquery.functions.validation.ValidationModule"/>           
            <module uri="http://exist-db.org/xquery/xmldb" class="org.exist.xquery.functions.xmldb.XMLDBModule"/>            

            <module uri="http://expath.org/ns/http-client" class="org.expath.exist.HttpClientModule"/>        
            <module uri="http://www.expath.org/mod/http-client" src="resource:org/expath/www/mod/http-client/http-client.xqm"/>      

            <module uri="http://exist-db.org/xquery/httpclient" class="org.exist.xquery.modules.httpclient.HTTPClientModule"/>

            <module uri="http://exist-db.org/xquery/kwic" src="resource:org/exist/xquery/lib/kwic.xql"/>

            <module uri="http://www.json.org" src="resource:org/exist/xquery/lib/json.xq"/>

            <module uri="http://exist-db.org/xquery/sequences" src="resource:org/exist/xquery/lib/sequences.xq"/>

            <module uri="http://exist-db.org/versioning" src="resource:org/exist/versioning/xquery/versioning.xqm"/>

            <module uri="http://exist-db.org/xquery/testing" src="resource:org/exist/xquery/lib/test.xq"/>   

            <module uri="http://xproc.net/xproc" src="resource:net/xproc/xprocxq/src/xquery/xproc.xqm"/>         
            <module uri="http://xproc.net/xproc/const" src="resource:net/xproc/xprocxq/src/xquery/const.xqm"/>          
            <module uri="http://xproc.net/xproc/ext" src="resource:net/xproc/xprocxq/src/xquery/ext.xqm"/>    
            <module uri="http://xproc.net/xproc/functions" src="resource:net/xproc/xprocxq/src/xquery/functions.xqm"/>        
            <module uri="http://xproc.net/xproc/naming" src="resource:net/xproc/xprocxq/src/xquery/naming.xqm"/>
            <module uri="http://xproc.net/xproc/opt" src="resource:net/xproc/xprocxq/src/xquery/opt.xqm"/> 
            <module uri="http://xproc.net/xproc/std" src="resource:net/xproc/xprocxq/src/xquery/std.xqm"/>     
            <module uri="http://xproc.net/xproc/util" src="resource:net/xproc/xprocxq/src/xquery/util.xqm"/>       

            <module uri="http://exist-db.org/xquery/file" class="org.exist.xquery.modules.file.FileModule"/>

            <module uri="http://exist-db.org/xquery/xqdoc" class="org.exist.xqdoc.xquery.XQDocModule"/>

        </builtin-modules>
    </xquery>
    <xupdate allowed-fragmentation="5" enable-consistency-checks="no"/>

</exist>

Is it a common problem? Or may be I configured eXist wrong?

Thanks for any help.

解决方案

Thanks to @joewiz and @avernet

Now I can answer my own question. The main problem is that I'm using orbeon 3.8, which use eXist DB to store cache there. Updating to 3.9 version will help, because 3.9 use Ehcahe instead.

As temporary workaround, I configured orbeon 3.8 to store data to external eXist for that in properties-local.xml I add

<property as="xs:anyURI"  name="oxf.fr.persistence.service.exist.uri" value="http://127.0.0.1:8080/exist/rest/db"/>

property as told here, and embeded eXist to store cache for that next properties should be set

<property as="xs:anyURI"  name="oxf.xforms.store.application.uri"                   value="xmldb:exist:///"/>                      
<property as="xs:string"  name="oxf.xforms.store.application.collection"            value="/db/orbeon/xforms/cache/"/>

So when servlet container stops I can safely delete embeded eXist data files.

这篇关于不断增长的 eXist 数据库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
其他开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆