使用elasticsearch-river-mysql将数据从MySQL数据库传输到Elasticsearch [英] Using elasticsearch-river-mysql to stream data from MySQL database to Elasticsearch
问题描述
我想使用 elasticsearch-river-mysql ，以便从MySQL数据库到ElasticSearch。我是ES和河流的初学者，所以我希望你能帮我解决问题。
I want to use elasticsearch-river-mysql in order to continuously transfer data from MySQL database to ElasticSearch. I'm beginner with ES and rivers so I hope you can help me out with my questions.
- 从我所知道的，数据将从MySQL数据库流式传输到ES集群，该集群将自动对其进行索引。那是对的吗？有什么超时或限制我必须注意吗？
- 关系数据库表之间的外键关系如何被翻译成ES？包含外键的表格行将成为ES文档的内部对象，还是使用ES文档之间的其他关系？
- 在使用此河时，是否有任何缺点以上目的？
- 通过实现这一点，ES上的负载是多少？我认为ES足够强大，但我仍然在想。在这种情况下，ES群集上的搜索是否会受到影响？
- From what I know, the data will be streamed from the MySQL database to the ES cluster which will index it automatically. Is that correct? Are there any timeouts or limits I have to be aware of?
- How the foreign key relations between the relational database tables will be translated into ES? Will the table row containing the foreign key become an inner object for an ES document or some other relation between the ES documents will be used?
- Are there any disadvantages in using this river for the mentioned above purpose?
- What will be the load on ES by implementing this? I assume that ES is powerful enough, but still I was wondering. Will the searches made on the ES cluster be affected anyhow in this scenario?
推荐答案
我的建议已经有很多原因试图使用 elasticsearch-jdbc-river 。
My advice already is to try to use the elasticsearch-jdbc-river for many reasons.
其中之一是，如果您决定切换 elasticsearch-jbdc-river > RDBMS 即可。
One of them is that the elasticsearch-jbdc-river is more generic in case you decide to switch RDBMS.
另一个原因是，另外一个还没有从2年起， jbdc-river 而弹性搜索从此开始发展很多。
Another is that the jbdc-river is still maintained when the other one hasn't been since 2 years, and Elasticsearch evolved a lot ever since.
1。从我所知道的，数据将从MySQL数据库流式传输到ES集群，该集群将自动对其进行索引。那是对的吗？是否有任何超时或限制我必须注意？
MySQL的数据应该从MySQL自动流式传输到Elasticsearch集群，而不会超时限制但瓶颈将是您的JVM堆大小。我不知道你需要多少处理你拥有的数据量。你需要测试它。
The data from MySQL should be streamed automatically from MySQL to the Elasticsearch cluster without a timeout limitation but the bottleneck will be your JVM Heap Size. I'm not sure how much do you need to process the amount of data you have. You need to test it.
2。关系数据库表之间的外键关系如何被翻译成ES？包含外键的表格行将成为ES文档的内部对象还是使用ES文档之间的其他关系？
弹性搜索是无模式的因此您需要管理到内部弹性搜索。河流将数据流传输到您的群集中。您可以在创建索引时定义映射，然后使用河流将其流式传输到ES群集。
Elasticsearch is schemaless so you need to manage to the inside Elasticsearch. The river just streams the data into your cluster. You can define your mapping when you create your index and then use the river to stream it into the ES cluster.
3。在上述目的使用这条河是否有任何缺点？
河流将被替换为另一种更清洁的方式来传输这些数据，但这是最好的解决方案。
The river will be replaced with another cleaner way to stream these data but this is the best solution you have for now.
这篇关于使用elasticsearch-river-mysql将数据从MySQL数据库传输到Elasticsearch的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
