ES到条形码数据传输 [英] ES to Crate data transfer

问题描述

我想将数据从ES（1.0.0）迁移到Crate（目前可用版本0.26.0），以便查看，我该怎么办？
我应该怎么处理索引，ES索引是否适合Crate？

可能会想到几个流程：

1. 平滑过渡，例如当我使用ES + Crate节点进行混合群集时，至少复制一个，然后逐个地杀死ES节点。


2. 我们正在通过'elasticsearch-cloud-aws'进行备份插件有可能还原这个快照，但是有了Crate节点而不是本机ES？

数据块，说1TB？

谢谢。

解决方案

Crate从0.24.0开始使用ES 1.0.1，所以lucene指数是兼容的。

这意味着你描述的平滑过渡流程应该起作用。 （但我
没有实际尝试过）。

但是，只要集群中有非包装箱节点，
条带箱上的SQL查询将无法正常工作，因为常规ES节点不知道如何处理
的请求。

您应该了解更多的事情可能会导致麻烦：

条件箱引入了0.25.0的显式数组类型。这意味着如果
创建一个新的表，例如创建表测试（id整数...） id字段
将使用添加到lucene的一些新的doc_values字段格式
前。如果要使用数组，则必须使用SQL（
类型定义，如 array（integer））创建表。

如果您尝试查询使用elasticsearch插入的数据，则可能会获得
奇怪的结果，甚至如果字段包含数组但
缺少模式定义中的声明。

希望这有所帮助。不幸的是，我不能给你一个关于
elasticsearch-cloud-aws插件的答案，但也许别人可以。

I want to migrate the data from ES (1.0.0) to Crate (currently available version 0.26.0) to check it out, how could I do it? What should I do with indexes, does ES index fit Crate?

One might think of several flows:

1. Smooth transition, e.g. when I do hybrid cluster with ES + Crate nodes, with replica at least one, and afterwards kill ES nodes one by one.
2. We're doing backups via 'elasticsearch-cloud-aws' plugin is it possible to 'restore' this snapshots, but having Crate nodes instead of native ES ones?

Any other variants to transfer big chunk of data, say 1TB?

Thanks.

解决方案

Crate is using ES 1.0.1 since 0.24.0 so the lucene indices are compatible.

This means that the "smooth transition" flow you described should work. (But I haven't actually tried it).

But as long as there are non-crate nodes in the cluster the SQL queries on crate won't work because regular ES nodes don't know how to handle the requests.

Some more things you should probably aware of as it could cause troubles:

Crate introduced explicit array types with 0.25.0. This means that If you create a new table with e.g. create table test (id integer ...) the id field will use the new doc_values field format that was added to lucene some time ago. If you want to use arrays you have to create the tables using SQL (with type definitions like array(integer) ).

If you try to query data that was inserted using elasticsearch you might get strange results or even run into errors if the fields contain arrays but are missing the declaration in the schema definition.

Hope this helps a bit. Unfortunately I can't give you an answer regarding the elasticsearch-cloud-aws plugin, but maybe someone else can.

这篇关于ES到条形码数据传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！