Java,Lucene:在Java中设置IndexWriter的锁定超时. [英] Java, Lucene : Set lock timeout for IndexWriter in Java.

查看:67
本文介绍了Java,Lucene:在Java中设置IndexWriter的锁定超时.的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在将Lucene与基于Spring-MVC的应用程序集成在一起.当前,我们可以使用它,但是很少会出现cannot obtain lock错误.之后,我必须手动删除锁定文件,然后它才能正常工作.

I am working on integrating Lucene with our Spring-MVC based application. Currently we have it working, but rarely we get a cannot obtain lock error. After which I have to manually delete the lock file and then it works normally.

如何为锁定Java中的索引设置超时?我没有针对Lucene的任何XML配置.我通过POM.xml在maven中添加了项目库,并实例化了所需的类.

How can I set a timeout for Locking the index in Java? I don't have any XML configuration for Lucene. I added the project library in maven via POM.xml and instantiated the required classes.

代码:

public void saveIndexes(String text, String tagFileName, String filePath, long groupId, boolean type, int objectId) {
        try {
            // path is the indexing directory. 
            File testDir;
            Path suggestionsPath;
            Directory suggestionsDir;

            Path phraseSuggestPath;
            Directory phraseSuggestDir;

            Directory directory = org.apache.lucene.store.FSDirectory.open(path);
            IndexWriterConfig config = new IndexWriterConfig(new SimpleAnalyzer());
            IndexWriter indexWriter = new IndexWriter(directory, config);

            org.apache.lucene.document.Document doc = new org.apache.lucene.document.Document();
            if (filePath != null) {
                File file = new File(filePath); // current directory
                doc.add(new TextField("path", file.getPath(), Field.Store.YES));
            }
            doc.add(new StringField("id", String.valueOf(objectId), Field.Store.YES));
            //  doc.add(new TextField("id",String.valueOf(objectId),Field.Store.YES));
            if (text == null) {
                if (filePath != null) {
                    FileInputStream is = new FileInputStream(filePath);
                    BufferedReader reader = new BufferedReader(new InputStreamReader(is));
                    StringBuilder stringBuffer = new StringBuilder();
                    String line;
                    while ((line = reader.readLine()) != null) {
                        stringBuffer.append(line).append("\n");
                    }
                    stringBuffer.append("\n").append(tagFileName);
                    reader.close();
                    doc.add(new TextField("contents", stringBuffer.toString(), Field.Store.YES));
                }
            } else {

                FieldType fieldType = new FieldType(TextField.TYPE_STORED);
                fieldType.setTokenized(false);
                doc.add(new Field("contents", text+"\n"+tagFileName, fieldType));
            }
            indexWriter.addDocument(doc);
            indexWriter.commit();
            indexWriter.flush();
            indexWriter.close();
            directory.close();

            StandardAnalyzer analyzer = new StandardAnalyzer();
            AnalyzingInfixSuggester wordSuggester = new AnalyzingInfixSuggester(suggestionsDir, analyzer);

            ArrayList<String> words = new ArrayList<>();
            if (text != null) {
                text = html2text(text);
                Pattern pt = Pattern.compile("[^\\w\\s]");
                Matcher match = pt.matcher(text);
                while (match.find()) {
                    String s = match.group();
                    text = text.replaceAll("\\" + s, "");
                }

                if (text.contains(" ")) {
                    Collections.addAll(words, text.split(" "));

                } else {
                    words.add(text);
                }
                SuggestionIterator suggestionIterator = new SuggestionIterator(words.iterator());
                wordSuggester.build(suggestionIterator);
                wordSuggester.close();
                suggestionsDir.close();
            }

            AnalyzingInfixSuggester phraseSuggester = new AnalyzingInfixSuggester(phraseSuggestDir, analyzer);
            if (text != null) {
                text = html2text(text);
                ArrayList<String> phrases = new ArrayList<>();
                phrases.add(text);
                SuggestionIterator suggestionIterator = new SuggestionIterator(phrases.iterator());
                phraseSuggester.build(suggestionIterator);
                phraseSuggester.close();
                phraseSuggestDir.close();
            }

        } catch (Exception ignored) {
        }
    }

谢谢.

推荐答案

我引用了

打开IndexWriter会为使用中的目录创建一个锁定文件. 尝试在同一目录上打开另一个IndexWriter将导致 一个LockObtainFailedException.

Opening an IndexWriter creates a lock file for the directory in use. Trying to open another IndexWriter on the same directory will lead to a LockObtainFailedException.

注意:IndexWriter实例是完全线程安全的,这意味着 多个线程可以同时调用其任何方法.如果你的 应用程序需要外部同步,您不应该 在IndexWriter实例上进行同步,因为这可能会导致死锁; 而是使用您自己的(非Lucene)对象.

NOTE: IndexWriter instances are completely thread safe, meaning multiple threads can call any of its methods, concurrently. If your application requires external synchronization, you should not synchronize on the IndexWriter instance as this may cause deadlock; use your own (non-Lucene) objects instead.

因此,如果IndexWriter已打开且未在其他位置关闭,则无法再次打开它.在您的情况下,当两个用户位于同一代码块中时,可能会遇到一些不幸的时机.

So you can't open IndexWriter again if its already opened and not closed somewhere else. In your case, there happens to be some unlucky timing when two users are in same code block.

您可以通过两种方式解决此问题,

You can address this issue in two ways ,

1. 指定关键部分::将具有编写者打开,使用和关闭操作的代码部分标记为关键部分,并在该关键部分上应用Java同步.使用某些应用程序Singleton bean进行同步.因此,当另一个用户点击该块时,他将等到第一个用户完成并释放锁定.

1.Designate Critical Section: Mark code portion having writer opening , usage and close operation as critical section and apply Java synchronization on that critical section. Use some app Singleton bean to synchronize on. So when another user hits that block, he will wait till first user is done and lock is released.

2. 单个编写器实例:在您的应用程序中开发一种机制,以在应用程序的生命周期内仅打开和关闭编写器一次,并在服务代码中传递该单个实例,以便可以通过以下方式调用writer方法: Lucene的人员将writer实例设为线程安全的,因此用户数量可以满足您的期望. 我想,这可以通过Singleton Spring bean并将该bean注入服务中来实现.

2.Single Writer Instance:Develop a mechanism in your app to open and close writer only once for the life time of application and pass that single instance in service code so writer methods could get called by as many users as you wish since writer instance is made thread-safe by Lucene folks. I guess, this can be achieved by a Singleton Spring bean and by injecting that bean in your service.

第二种方法的缺点是-单个全局索引目录的多服务器部署,以及是否有其他应用程序试图在该全局索引上打开写入器.可以通过将索引编写器实例创建代码包装在某种全局服务中来解决此问题,该代码始终将相同的实例返回给尝试使用它的任何应用程序.

Drawback in second approach is - multi server deployments for a single global index directory and if there are other applications trying to open writers on that global Index. This problem can be solved by wrapping your index writer instance creation code in some kind of global service that keeps returning the same instance to whichever application tries to use it.

这不是您要通过删除锁定文件或引入超时来解决的简单问题.您必须根据IndexWriter文档对设计进行建模,而不要采用其他方法.

This is not a simple issue that you are trying to solve by deleting lock files or by introducing time outs. You have to model your design as per IndexWriter documentation and not other way round.

具有单个writer实例也会带来一些性能改进.

Having single writer instance will introduce some performance improvements too.

此外,在创建编写器后立即进行一次空提交.这帮助我解决了过去的一些问题.

Also, make a practice to do an empty commit just after creating the writer. This helped me in solving some issues in past.

这篇关于Java,Lucene:在Java中设置IndexWriter的锁定超时.的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆