JVM在Lucene DataInput.readVInt上崩溃 [英] JVM crashes on Lucene DataInput.readVInt

问题描述

在使用Lucene索引文档时，我的JVM（1.6.0_29）在密集使用时不断崩溃。
我得到：

 ＃
＃Java运行时环境检测到致命错误：
＃
＃SIGSEGV（0xb）at pc = 0x00002b6b196d767c，pid = 26417，tid = 1183217984 
＃
＃JRE版本：6.0_29-b11 
＃Java VM： Java HotSpot（TM）64位服务器VM（20.4-b02混合模式linux-amd64压缩oops）
＃有问题的框架：
＃J org.apache.lucene.store.DataInput.readVInt（）I 
＃
＃如果您想提交错误报告，请访问：
＃http://java.sun.com/webapps/bugreport/crash.jsp 
＃ 
  

环境：

JDK：1.6u29 （与1.6_02相同的问题）
Lucene版本3.4.0

vm_info：适用于Linux的Java HotSpot（TM）64位服务器VM（20.4-b02） -amd64 JRE（1.6.0_29-b11），建于2011年10月3日01:19:20java_re与gcc 3.2.2（SuSE Linux）

OS ：CentOS 5.0版（最终版）

jvm_args：-Dcatalina.home = / var / local / tomcat-8081 -Dcatalina.base = / var / local / to mcat-8081 -Djava.io.tmpdir = / var / tmp -Dfile.encoding = UTF-8 -Xmx1024M -XX：MaxPermSize = 96m

它似乎是一个在jdk 1.7中修复的jdk问题，但引入的其他问题。
https://issues.apache.org/jira/browse/LUCENE-3335
Java 7包含自1.6.0_21以来对readVInt问题的修复（约，LUCENE-2975）

所以，我怎么能使用JDK 1.6修复此问题？
我应该升级到jdk 1.7吗？

解决方案

这些JDK问题也在1.6.9_29中修复（不仅仅是1.7 .0u1）。 ReadVInt不再崩溃。因此，您的崩溃与任何着名的java6 / 7错误无关（vint错误不会导致您的JVM崩溃，只会通过返回错误值来破坏您的索引 - 而且这个版本自Lucene 3.1以来肯定已经修复）。 / p>

但是你可能会崩溃你的JVM：你是64位平台（Linux），因此默认目录实现是MMapDirectory。 Lucene使用hack能够从虚拟地址空间取消映射映射文件。 JVM本身不允许这样做，但是取消依赖垃圾收集器，这对Lucene来说是一个问题。默认情况下，MMapDirectory在关闭IndexInputs后取消映射文件。 MMapDirectory根本没有同步，所以当另一个线程在取消映射后尝试访问IndexInput时，它将访问一个未映射的地址并且将是SIGSEGV。

如果你的代码是正确的，这个不可能发生，但看起来你正在使用已经关闭的IndexReader / IndexWriter来访问索引。在Lucene 3.5（即将推出）之前，在IndexReader中丢失检查将使已经关闭的IndexReader及其所有已关闭（和未映射）的IndexInputs尝试访问索引数据和段错误成为可能。

在3.5中，我们添加了额外的安全检查以防止此非法访问，但它不是100％（因为缺少同步）。我会查看代码并检查没有任何内容访问封闭索引。

一个简单的检查，看看这是否是你的问题将是使用NIOFSDirectory（在Linux上较慢）而不是MMapDirectory。如果它没有崩溃并且可能抛出AlreadyClosedExceptions，则该错误正在访问已关闭的索引。

My JVM (1.6.0_29) keeps crashing on intensive use when indexing documents with Lucene. I get:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00002b6b196d767c, pid=26417, tid=1183217984
#
# JRE version: 6.0_29-b11
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.4-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J  org.apache.lucene.store.DataInput.readVInt()I
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

Environment:

JDK: 1.6u29 (same issue with 1.6_02) Lucene Version 3.4.0

vm_info: Java HotSpot(TM) 64-Bit Server VM (20.4-b02) for linux-amd64 JRE (1.6.0_29-b11), built on Oct 3 2011 01:19:20 by "java_re" with gcc 3.2.2 (SuSE Linux)

OS:CentOS release 5.0 (Final)

jvm_args: -Dcatalina.home=/var/local/tomcat-8081 -Dcatalina.base=/var/local/tomcat-8081 -Djava.io.tmpdir=/var/tmp -Dfile.encoding=UTF-8 -Xmx1024M -XX:MaxPermSize=96m

It seems to be a jdk issue that was fixed in jdk 1.7, but other issues where introduced. https://issues.apache.org/jira/browse/LUCENE-3335 "Java 7 contains a fix to the readVInt issue since 1.6.0_21 (approx, LUCENE-2975)"

So, how can I fix this issue using JDK 1.6? Should I upgrade to jdk 1.7?

解决方案

these JDK issues are also fixed in 1.6.9_29 (not only 1.7.0u1). ReadVInt can no longer crash. So your crash is not related to any of the "famous java6/7 bugs" (the vint bug does not crash your JVM at all it just corrupts your index by returning wrong values - and this one is definitely fixed since Lucene 3.1).

But there is another chance you can crash your JVM: You are on a 64 bit platform (Linux), so the default directory implementation is MMapDirectory. Lucene uses a hack to be able to unmap mapped files from virtual address space. This is not allowed by the JVM itsself, but makes unmapping dependent on garbage collector, which is a problem for Lucene. By default MMapDirectory unmaps the files after closing the IndexInputs. MMapDirectory is not synchronized at all, so when another thread tries to access the IndexInput after unmapping it will access an unmapped address and will SIGSEGV.

If your code would be correct this cannot happen, but it looks like you are using an already closed IndexReader/IndexWriter to access the index. Before Lucene 3.5 (will come out soon), missing checks in IndexReader will make it possible that an already closed IndexReader with all its closed (and unmapped) IndexInputs tries to access index data and segfaults.

In 3.5 we added additional safety checks to prevent this illegal access, but its not 100% (as synchronization is missing). I would review the code and check that nothing accesses closed index.

A simple check to see if this is your issue would be to use NIOFSDirectory (slower on Linux) instead of MMapDirectory. If it does not crash and possibly throws AlreadyClosedExceptions, the bug is accessing closed indexes.

这篇关于JVM在Lucene DataInput.readVInt上崩溃的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！