在MongoDB和Apache Solr之间同步数据的简便方法 [英] Easy way to Sync Data between MongoDB and Apache Solr

问题描述

我最近开始使用MongoDB和Apache Solr。我使用MongoDB作为数据存储，我希望Apache Solr为我的应用程序中的搜索功能创建数据索引。

经过一些研究我发现，那里基本上是两种方法在MongoDB和Solr之间同步数据。

1）使用Solr DataImportHandler -

For我使用了由james创建的SolrMongoImporter，并按照他的教程 github

我能够成功运行Import Handler并且Solr识别出ImportHandler，但它没有将任何文档导入solr。每次它说更新文件= 0。

2）然后我尝试切换到MongoDB端，看看是否存在任何东西，我发现
MongoDBConnector 由10gen提供。

当我按照说明操作并运行连接器时，它会尝试将大量文档发布到Solr，并提供以下输出。

  2012-11-24 15：15：20,665  -  INFO  - 完成'http：// localhost：8983 / solr / update /？commit = true'（ POST）与主体'< commit />'在0.010秒内。 
 2012-11-24 15：15：21,674  -  INFO  - 完成'http：// localhost：8983 / solr / update /？commit = true'（POST）with body'< commit />'in 0.009秒。 
 2012-11-24 15：15：22,683  -  INFO  - 已完成'http：// localhost：8983 / solr / update /？commit = true'（POST）与body'< commit />'in 0.008秒
 2012-11-24 15：15：23,694  -  INFO  - 完成'http：// localhost：8983 / solr / update /？commit = true'（POST）with body'< commit />'in 0.010秒
 2012-11-24 15：15：24,702  -  INFO  - 已完成'http：// localhost：8983 / solr / update /？commit = true'（POST）与body'< commit />'in 0.008秒
 2012-11-24 15：15：25,711  -  INFO  - 已完成'http：// localhost：8983 / solr / update /？commit = true'（POST）with body'< commit />'in 0.008秒
 2012-11-24 15：15：26,722  -  INFO  - 完成'http：// localhost：8983 / solr / update /？commit = true'（POST）with body'< commit />'in 0.010秒
  

但Solr中没有数据。

<我想知道哪种方法对你们有用，有没有关于MongoDB和Solr Integration的好教程。

另外，我正在寻找一个实时的在MongoDB和solr之间同步，即只要将任何产品添加到我的mongodb，我希望它在solr索引中更新并反映在搜索结果中。

我正在使用MongoDB 2.0.4和Solr 3.6.1。

解决方案

Hadoop是一个创建SOLR索引的选项。我没有做过这方面的第一手资料，但是已经听取了人们的意见 as etsy 是谁。

关于此课程在lucene革命中，他们谈到了使用hadoop更新某些SOLR核心中的索引。不幸的是，我不认为课程材料是公开的。

并且这个演讲演讲者谈到了mongo / hadoop支持。

其他相关链接：

• 通过Solr和Java MapReduce索引文件


• 使用Hadoop创建Solr索引


• Mongo-Hadoop Connector

I recently started working with MongoDB and Apache Solr. I am using MongoDB as a data store and I want Apache Solr to create index for my data for the search feature in my application.

After some research I found out, there are basically 2 methods to sync the data between MongoDB and Solr.

1) using Solr DataImportHandler -

For this I used SolrMongoImporter created by james and followed his tutorial on github

I was able to successfully run the Import Handler and Solr identified the ImportHandler but it was not importing any documents into solr. Every time it said updated documents=0.

2) Then I tried switching to MongoDB side, to look if anything exists there and I found MongoDBConnector provided by 10gen.

When I followed the instructions, and ran the connector, it is trying to post lot of documents to Solr and it gives the following output.

2012-11-24 15:15:20,665 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.010 seconds.
2012-11-24 15:15:21,674 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.009 seconds.
2012-11-24 15:15:22,683 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.008 seconds.
2012-11-24 15:15:23,694 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.010 seconds.
2012-11-24 15:15:24,702 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.008 seconds.
2012-11-24 15:15:25,711 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.008 seconds.
2012-11-24 15:15:26,722 - INFO - Finished 'http://localhost:8983/solr/update/?commit=true' (POST) with body '<commit />' in 0.010 seconds.

But no data is there in Solr.

I wanted to know which approach worked for you guys, and is there any good tutorial on MongoDB and Solr Integration.

Also, I am looking for a real-time sync between MongoDB and solr, i.e. as soon as any product is added to my mongodb, I want it updated in solr index and reflect in search results.

I am using MongoDB 2.0.4 and Solr 3.6.1.

解决方案

Hadoop is an option for creating SOLR indexes. I haven't done this first hand, but have heard from people such as etsy who are.

On this course at lucene revolution they talked about using hadoop to update the indexes in some SOLR cores. Unfortunately I don't think the course material is publicly available.

And at this talk the speaker talked about the mongo/hadoop support.

Other related links:

• Indexing Files via Solr and Java MapReduce
• Using Hadoop to Create Solr Indexes
• Mongo-Hadoop Connector

这篇关于在MongoDB和Apache Solr之间同步数据的简便方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！