Cassandra数据从1.2迁移到3.0.2 [英] Cassandra data Migration from 1.2 to 3.0.2
问题描述
我知道类似的问题以前被问过，但我认为我的用例是非常具体的，我找不到任何答案。
在生产中，我们使用Cassandra 1.2与ByteOrderPartitioner在6节点集群中使用Priam作为种子管理工具。我们最近升级了所有依赖项，并试图迁移到Cassandra 3.0.2与Murmur分区程序，为了向后兼容性，我们需要在新集群上启用节俭。我们还要迁移离开Priam。
我能够设置新的集群，但在数据迁移期间面临很多问题。我试了3件事：
1）使用复制命令：当行数很大时失败
SSTable2Json：Cassandra 3.0.2已停止支持SSTable2Json
3）SSTableloader：因为不同的源和目标的cassandra版本，我认为失败
java.lang.RuntimeException：无法检索端点范围：
at org.apache.cassandra.tools.BulkLoader $ ExternalClient.init（BulkLoader.java:233）
at org。 apache.cassandra.io.sstable.SSTableLoader.stream（SSTableLoader.java:119）
at org.apache.cassandra.tools.BulkLoader.main（BulkLoader.java:67）
引发者：InvalidRequestException为什么：unconfigured表schema_columnfamilies）
在org.apache.cassandra.thrift.Cassandra $ execute_cql3_query_result.read（Cassandra.java:37849）
在org.apache.thrift.TServiceClient.receiveBase（TServiceClient.java： 78）
at org.apache.cassandra.thrift.Cassandra $ Client.recv_execute_cql3_query（Cassandra.java:1562）
at org.apache.cassandra.thrift.Cassandra $ Client.execute_cql3_query（Cassandra.java： 1547）
at org.apache.cassandra.tools.BulkLoader $ ExternalClient.init（BulkLoader.java:225）
...另外2个
现在我有点卡住，任何帮助，这将深深感激。如果您需要更多详细信息，请告诉我。
否，您不能将sstables 从1.2升级到3.0.2 ，因为sstable将不同的版本。此 链接 说明升级cassandra版本的步骤。但是它也对您没有帮助，因为您在分区器类型中有更改。
更改分区类型在cassandra中不支持
now （链接） 。
我更喜欢的解决方案之一是
创建一个独立的实用程序，它是cassandra 3.0.2版本，从你的源cassandra的数据和写入sstable
与 CQLSSTableWriter ，分区类型为Murmur Partitioner（诀窍是，你正在使用3.0.2版本编写
sstable，所以这个sstable将会容易
被您的新群集识别）。然后在目标群集中使用SSTableLoader
但我不知道为什么你还是需要向后兼容性，而创建CQLSSTableWritter时，您可以使用关键字
WITH COMPACT STORAGE指定列系列模式。但我没有尝试CQLSSTableWritter与COMPACT STORAGE，但没有COMPACT STORAGE我试过，它将适用于您的情况。
I know similar questions have been asked before but I think my use case is very specific for which I could not find any answer.
In Production we are using Cassandra 1.2 with ByteOrderPartitioner in a 6 Node cluster with Priam as seed management tool. We have recently upgraded all the dependencies and trying to migrate to Cassandra 3.0.2 with Murmur Partitioner and for backward compatibility we need to enable thrift on new cluster .Also we want to migrate away from Priam also. I was able to setup new cluster but facing lot of issues during data migration. I tried 3 things:
1) Use Copy Command : Fails when number of rows is large
2) SSTable2Json : Cassandra 3.0.2 has stopped supporting SSTable2Json
3) SSTableloader: Failing I think because of different cassandra version of source and destination
java.lang.RuntimeException: Could not retrieve endpoint ranges: at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:233) at org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:119) at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:67) Caused by: InvalidRequestException(why:unconfigured table schema_columnfamilies) at org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:37849) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1562) at org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1547) at org.apache.cassandra.tools.BulkLoader$ExternalClient.init(BulkLoader.java:225) ... 2 more
Right now I am kind of stuck,any help regarding this will be deeply appreciated. Please let me know if you need more details.
No, You cannot upgrade your sstables from 1.2 to 3.0.2 directly since the sstable will differ for different version. This link describes the steps for upgrading the cassandra versions. But it also does not helps for you, since you are having a change in the partitioner type.
Changing the partitioner type is not yet supported in cassandra as of now (Link).
One of the solution I would prefer is,
Create a stand alone utility which is of cassandra 3.0.2 version to read all the data from you source cassandra and write to sstable with the help of CQLSSTableWriter with the partition type of Murmur Partitioner (The trick is, you are writing the sstable with the version 3.0.2, so this sstable will be easily recognized by your new cluster). Then use SSTableLoader in your target cluster
But I am not sure about why you still require backward compatibility, while creating CQLSSTableWritter you can specify the column family schema with keyword "WITH COMPACT STORAGE". But I didn't tried CQLSSTableWritter with "WITH COMPACT STORAGE", but without "WITH COMPACT STORAGE" I had tried, it will work for your case too.
这篇关于Cassandra数据从1.2迁移到3.0.2的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
