简单使用 Solr 时如何解决“锁定获取超时"? [英] How to solve the 'Lock obtain timed out' when using Solr plainly?

问题描述

我的 Solr 系统有两个内核(Solr 版本 3.6.1).当我在我们的专用 Solr 服务器上调用以下命令行来添加并索引文件时:

java -Durl=http://solrprod:8080/solr/original/update -jar/home/solr/solr3/biomina/solr/post.jar/home/solr/tmp/2008/c2m-dump-01.noDEID_clean.xml

/usr/share/tomcat7/logs/solr.2013-12-11.log 文件中出现异常(大约等待 6 分钟后):

严重:org.apache.lucene.store.LockObtainFailedException:锁获取超时:NativeFSLock@/home/solr/solr3/biomina/solr/original/data/index/write.lock

(您可以在此消息的末尾看到它的详细输出).

我尝试修改锁的超时时间(通过将 writeLockTimeout 设置为 300000)，但这并没有解决问题.我没有使用任何自定义脚本，只是使用 Solr 3.1.6 附带的 post.jar 来添加和索引.

关于需要更改哪些内容以消除此错误并成功将 XML 文件添加到 Solr 并对其进行索引有什么想法吗?

/home/solr/solr3/biomina/solr/solr.xml的内容:

<?xml version="1.0" encoding="UTF-8" ?><!--获得 Apache 软件基金会 (ASF) 的一项或多项许可贡献者许可协议.请参阅分发的 NOTICE 文件这项工作以获取有关版权所有权的更多信息.ASF 根据 Apache 许可证 2.0 版将此文件许可给您(许可证")；除非符合以下规定，否则您不得使用此文件许可证.您可以在以下网址获取许可证的副本http://www.apache.org/licenses/LICENSE-2.0除非适用法律要求或书面同意，否则软件根据许可分发是在原样"基础上分发的，没有任何明示或暗示的保证或条件.请参阅许可证以了解特定语言的管理权限和许可证下的限制.--><!--所有(相对)路径都相对于安装路径持久性:将通过 API 所做的更改保存到此文件sharedLib:将在所有内核之间共享的 lib 目录的路径--><solr 持久化="true"><!--adminPath:管理核心的 RequestHandler 路径.如果为null"(或不存在)，则无法通过请求处理程序管理核心--><cores adminPath="/admin/cores"><core name="original" instanceDir="original"/><core name="deidentified" instanceDir="deidentified"/></核心></solr>

solrconfig.xml 的相关部分(对于名为 original 的核心):

<!-- maxFieldLength 指定每个索引的 *tokens* 的最大数量场地.默认值:10000 --><!-- <maxFieldLength>10000</maxFieldLength>--><!-- 等待 IndexWriter 写入锁定的最长时间(毫秒).默认值:1000 --><writeLockTimeout>300000</writeLockTimeout>

solrconfig.xml 的相关部分(对于名为 deidentified 的核心):

<!-- maxFieldLength 指定每个索引的 *tokens* 的最大数量场地.默认值:10000 --><!-- <maxFieldLength>10000</maxFieldLength>--><!-- 等待 IndexWriter 写入锁定的最长时间(毫秒).默认值:1000 --><writeLockTimeout>300000</writeLockTimeout>

异常的详细输出

2013 年 12 月 11 日上午 11:27:25 org.apache.solr.core.SolrCore 执行INFO:[原始] webapp=/solr path=/update params={} status=500 QTime=3000702013 年 12 月 11 日上午 11:32:25 org.apache.solr.common.SolrException 日志严重:org.apache.lucene.store.LockObtainFailedException:锁定获取超时:NativeFSLock@/home/solr/solr3/biomina/solr/original/data/index/write.lock在 org.apache.lucene.store.Lock.obtain(Lock.java:84)在 org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)在 org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:84)在 org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:101)在 org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:171)在 org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:219)在 org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)在 org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115)在 org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:157)在 org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79)在 org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58)在 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)在 org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)在 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)在 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)在 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)在 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)在 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)在 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)在 org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)在 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)在 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)在 org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)在 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)在 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)在 org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)在 org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)在 org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156)在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:626)在 java.lang.Thread.run(Thread.java:804)2013 年 12 月 11 日上午 11:32:25 org.apache.solr.core.SolrCore 执行INFO:[原始] webapp=/solr path=/update params={} status=500 QTime=556916

系统详情:

uname -aLinux solrprod 3.0.93-0.8-default #1 SMP Tue Aug 27 08:44:18 UTC 2013 (70ed288) x86_64 x86_64 x86_64 GNU/Linuxjava版本java版本1.7.0"Java(TM) SE 运行时环境(构建 pxa6470sr6-20131015_01(SR6))IBM J9 VM(build 2.6，JRE 1.7.0 Linux amd64-64 Compressed References 20131013_170512(启用 JIT，启用 AOT)J9VM - R26_Java726_SR6_20131013_1510_B170512JIT - r11.b05_20131003_47443GC - R26_Java726_SR6_20131013_1510_B170512_CMPRSSJ9CL-20131013_170512)JCL - 20131011_01 基于 Oracle 7u45-b18

解决方案

以下修改解决了这个问题:

• 应用了 https://stackoverflow.com/a/3035916/236007 中描述的更改p>

• 切换到 Oracle Java 运行时(它是 IBM Java 运行时).

• 把ulimit -v unlimited放到/etc/init.d/tomcat7中.

• 修改/usr/share/tomcat7/bin/setenv.sh文件如下(给它大约4 GB内存):

• <代码>export JAVA_OPTS="$JAVA_OPTS -Xmx4000m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/mnt/data/tomcat_dump"

I have two cores for our Solr system (Solr version 3.6.1). When I invoke the following command line on our dedicated Solr server to add and then index a file:

java -Durl=http://solrprod:8080/solr/original/update -jar /home/solr/solr3/biomina/solr/post.jar /home/solr/tmp/2008/c2m-dump-01.noDEID_clean.xml

I get an exception in /usr/share/tomcat7/logs/solr.2013-12-11.log file (after about 6 minutes of waiting):

SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/home/solr/solr3/biomina/solr/original/data/index/write.lock

(You can see the detailed output of it at the end of this message).

I tried to modify the time-out for locks (by setting writeLockTimeout to 300000) , but this did not solve the problem. I'm not using any custom script, just the post.jar that comes with Solr 3.1.6, to add and index.

Any ideas about what needs to be changed to get rid of this error and successfully add the XML file about to Solr and index it?

Contents of /home/solr/solr3/biomina/solr/solr.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<!--
 All (relative) paths are relative to the installation path

  persistent: Save changes made via the API to this file
  sharedLib: path to a lib directory that will be shared across all cores
-->
<solr persistent="true">

  <!--
  adminPath: RequestHandler path to manage cores.  
    If 'null' (or absent), cores will not be manageable via request handler
  -->
  <cores adminPath="/admin/cores">
    <core name="original" instanceDir="original" />
    <core name="deidentified" instanceDir="deidentified" />
  </cores>
</solr>

Relevat part of solrconfig.xml (for the core named original):

<indexConfig>
    <!-- maxFieldLength specifies max number of *tokens* indexed per 
field. Default: 10000 -->
    <!-- <maxFieldLength>10000</maxFieldLength>  -->
    <!-- Maximum time to wait for a write lock (ms) for an IndexWriter. 
Default: 1000 -->
    <writeLockTimeout>300000</writeLockTimeout>

Relevat part of solrconfig.xml (for the core named deidentified):

<indexConfig>
    <!-- maxFieldLength specifies max number of *tokens* indexed per 
field. Default: 10000 -->
    <!-- <maxFieldLength>10000</maxFieldLength>  -->
    <!-- Maximum time to wait for a write lock (ms) for an IndexWriter. 
Default: 1000 -->
    <writeLockTimeout>300000</writeLockTimeout>

Detailed Output of Exception

Dec 11, 2013 11:27:25 AM org.apache.solr.core.SolrCore execute
INFO: [original] webapp=/solr path=/update params={} status=500 QTime=300070 
Dec 11, 2013 11:32:25 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/home/solr/solr3/biomina/solr/original/data/index/write.lock
    at org.apache.lucene.store.Lock.obtain(Lock.java:84)
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1098)
    at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:84)
    at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:101)
    at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:171)
    at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:219)
    at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)
    at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115)
    at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:157)
    at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:626)
    at java.lang.Thread.run(Thread.java:804)

Dec 11, 2013 11:32:25 AM org.apache.solr.core.SolrCore execute
INFO: [original] webapp=/solr path=/update params={} status=500 QTime=556916

System details:

uname -a

Linux solrprod 3.0.93-0.8-default #1 SMP Tue Aug 27 08:44:18 UTC 2013 (70ed288) x86_64 x86_64 x86_64 GNU/Linux

java -version

java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470sr6-20131015_01(SR6))
IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 Compressed References 20131013_170512 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR6_20131013_1510_B170512
JIT  - r11.b05_20131003_47443
GC   - R26_Java726_SR6_20131013_1510_B170512_CMPRSS
J9CL - 20131013_170512)
JCL - 20131011_01 based on Oracle 7u45-b18

解决方案

The following modifications solved the issue:

• Applied the changes described at https://stackoverflow.com/a/3035916/236007

• Switched to Oracle Java runtime (it was IBM Java runtime).

• Put the ulimit -v unlimited in /etc/init.d/tomcat7.

• Modified the /usr/share/tomcat7/bin/setenv.sh file as the following (giving it about 4 GB memory):

• export JAVA_OPTS="$JAVA_OPTS -Xmx4000m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/mnt/data/tomcat_dump"

这篇关于简单使用 Solr 时如何解决“锁定获取超时"?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！