Elasticsearch-从2.x升级到7.x-使用TransportClient和Jest进行双重写入 [英] Elasticsearch - Upgrading from 2.x to 7.x - Dual write using TransportClient and Jest

问题描述

我们正在从elasticsearch 2.x迁移到7.x，方法是双重写入两个集群.稍后，以2.x至5.x，然后从5.x至6.x，最后从6.x至7.x的多个跃点将快照从2.x还原到7.x.

现在，问题在于启用对旧集群和新集群的双重写入，我们使用TransportClient(tcp连接)和Jest(http连接)来索引文档，是否有任何客户端可以处理此问题?据我所知，Jest不支持7.x，在这里我们有什么选择?

编辑

我们正在使用Kafka connect将相同的更新发布到7.x elasticsearch.寻找非kafka解决方案的人仍然可以解决这个问题.

解决方案

您是对的JEST不支持7.X，Elasticsearch官方客户端不适用于2.X :)到目前为止，尚无客户端可以写入这两个版本，即使Elasticsearch客户端随其服务器版本一起发布，并且在主要版本发生更改时也不兼容.

此外，当您使用Java时，我强烈建议您使用官方ES博客，很快将弃用运输客户，而优先选择高级客户.同一博客的文字:

高级客户端最终将取代未来，因此我们邀请所有Java用户尝试一下，如果可能的话，迁移到它.当前版本支持以下内容Elasticsearch API:

解决方案:

您需要使用maven shade插件来重新定位 jest 或 Elasticsearch客户端中存在的类，因为它们具有冲突的类，因此您可以同时使用客户，并使用它们，您可以索引和查询其Elasticsearch服务器.

可以在此处找到有关阴影插件的更多信息.和此处.

We are in process of migrating from elasticsearch 2.x to 7.x, the approach is to dual write to both the clusters. Later in time, restore the snapshot from 2.x to 7.x in multiple hops of 2.x to 5.x then 5.x to 6.x and finally 6.x to 7.x.

Now, the issue is enabling dual write to old and new clusters, we are using TransportClient (tcp connection) and Jest (http connection) to index documents, is there any client which can handle this? As far as I know, Jest doesn't have support for 7.x, what are our choices here?

EDIT

We are using Kafka connect to ship the same updates to 7.x elasticsearch. This question is still open for people looking for non-kafka solutions.

解决方案

You are right JEST doesn't support the 7.X and Elasticsearch official clients are not available for 2.X :) and till now there is no client which can write to both the versions, even Elasticsearch clients are released with its server version and are not compatible when there is a change in the major version.

Also, As you are using Java, I would highly recommend using the high-level client, instead of transport the client. As mentioned in official ES blog, soon transport clients would be deprecated in favor of the high-level clients. text from the same blog:

The high-level client will eventually replace the transport client in the future, hence we invite all our Java users to try it out and migrate to it if possible. The current release supports the following Elasticsearch APIs:

Solution:

You need to use maven shade plugin to relocate the class exist in either of your jest or Elasticsearch client as they have conflicting classes and this way you can use both the clients and using them you can index and query their Elasticsearch servers.

More info on shade plugins can be found here and here.

这篇关于Elasticsearch-从2.x升级到7.x-使用TransportClient和Jest进行双重写入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！