Microsoft Cosmosdb for Mongodb:将未分片的集合合并为分片的集合 [英] Microsoft Cosmosdb for Mongodb: merge unsharded collection into sharded ones

问题描述

我有 2 个类似文档的集合(即相同的对象，不同的值).一个集合(X)在数据库 A 中未分片，另一个集合(Y)在数据库 B 中分片.当我尝试将集合 X 复制到数据库 B 时，我收到错误消息，提示共享吞吐量集合应该有一个分区键".我也试过用 foreach insert 复制数据，但是时间太长了.

I have 2 collections of similar documents(i.e. same object, different values). One collection(X) is unsharded in database A, another collection(Y) is sharded and inside database B. When I try copy collection X into database B, I got error saying that "Shared throughput collection should have a partition key". I also tried copying data using foreach insert, but it takes too long time.

所以我的问题是，如何以有效的方式将数据从集合 X 附加到集合 Y?

So my question is, how can I append the data from collection X to collection Y in efficient way?

CosmosDB 上的 Mongodb 版本是 3.4.6

Mongodb version on CosmosDB is 3.4.6

推荐答案

因为 CosmosDB 中的 MongoDb 版本目前是 3.4.6，不支持 $merge 和很多其他命令，如 colleciton.copyTo 等.使用Studio 3T 的导入功能也没有帮助.

Because, the MongoDb version in CosmosDB currently 3.4.6, it doesn't support $merge and a lot of other commands such as colleciton.copyTo etc. Using Studio 3T's import feature didn't help as well.

我使用的解决方案是在我的本地 mongodb 上下载目标集合，对其进行清理，然后编写 Java 代码，该代码将从本地数据库读取我的清理数据并将其插入(或批量写入)到目标集合.这样，数据将被附加到目标集合.我测量的速度是 100 万个文档(~750MB)的 2 小时，当然，这个数字可能会因各种因素而异，例如网络、文档大小等.

The solution I use, is to download the target collection on my local mongodb, clean it then write java code that will read my clean data from local db and insertMany(or bulkwrite) it to the target collection. This way, the data will be appended to the target collection. The speed I measured was 2 hours for 1m document count(~750MB), of course, this numbers might vary depending on various factors, i.e. network, document size etc.

这篇关于Microsoft Cosmosdb for Mongodb:将未分片的集合合并为分片的集合的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！