cosmosdb-将超过n年的数据存档到冷存储中 [英] cosmosdb - archive data older than n years into cold storage

问题描述

我研究了几个地方，找不到将cosmosdb中的旧数据存档到冷存储器中的选择的任何方向.我在AWS中看到了DynamoDb，其中提到您可以将dynamodb数据移动到S3中.但不确定cosmosdb有哪些选项.我知道有生存时间选项，该数据将在特定日期后删除数据，但是我对存档还是删除感兴趣.任何方向将不胜感激.谢谢

I researched several places and could not find any direction on what options are there to archive old data from cosmosdb into a cold storage. I see for DynamoDb in AWS it is mentioned that you can move dynamodb data into S3. But not sure what options are for cosmosdb. I understand there is time to live option where the data will be deleted after certain date but I am interested in archiving versus deleting. Any direction would be greatly appreciated. Thanks

推荐答案

我认为CosmosDB中没有单击内置的功能即可实现.

I don't think there is a single-click built-in feature in CosmosDB to achieve that.

还是，正如您提到的了解任何方向一样，那么我建议您考虑

Still, as you mentioned appreciating any directions, then I suggest you consider DocumentDB Data Migration Tool.

有关数据迁移工具的注意事项:

Notes about Data Migration Tool:

• 您可以指定查询以仅提取冷数据(例如，按存储在文档中的创建日期).
• 支持将导出导出到各种目标(JSON文件，blob 存储，数据库，另一个cosmosDB集合等.)，
• 压缩过程中的数据-可以将文档合并到单个数组文档中并进行压缩.
• 设置好配置后，就可以编写脚本了 使用您喜欢的计划工具自动触发.
  • 您可以轻松地反转源和目标，以将冷数据恢复到活动存储(或恢复到开发，测试，备份等).
  • you can specify a query to extract only the cold-data (for example, by creation date stored within documents).
  • supports exporting export to various targets (JSON file, blob storage, DB, another cosmosDB collection, etc..),
  • compacts the data in the process - can merge documents into single array document and zip it.
  • Once you have the configuration set up you can script this to be triggered automatically using your favorite scheduling tool.
    • you can easily reverse the source and target to restore the cold data to active store (or to dev, test, backup, etc).

    要删除导出的数据，可以使用上述的TTL功能，但是如果导出步骤失败，则可能导致数据丢失.我建议编写并执行一个存储过程，以通过一次调用查询和删除所有导出的文档.该SP不会自动执行，但可以包含在自动化脚本中，并且仅在成功导出数据后才能执行. 请参阅: Azure Cosmos DB服务器端编程:存储过程，数据库触发器，和UDFs .

    To remove exported data you could use the mentioned TTL feature, but that could cause data loss should your export step fail. I would suggest writing and executing a Stored Procedure to query and delete all exported documents with single call. That SP would not execute automatically but could be included in the automation script and executed only if data was exported successfully first. See: Azure Cosmos DB server-side programming: Stored procedures, database triggers, and UDFs.

    这篇关于cosmosdb-将超过n年的数据存档到冷存储中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！