在集合之间移动文档是表示 MongoDB 中状态变化的好方法吗? [英] Is moving documents between collections a good way to represent state changes in MongoDB?

问题描述

我有两个集合，一个 (A) 包含要处理的项目(相对较小)，一个 (B) 包含已处理的项目(相当大，有额外的结果字段).

从A 读取项目，进行处理并保存() 到B，然后从A 中删除().

基本原理是这些索引可以不同，并且传入"集合可以通过这种方式保持非常小和快速.

我遇到了两个问题:

• 如果 remove() 或 save() 超时或加载失败，我将完全丢失该项目，或处理它两次
• 如果两者都失败，副作用就会发生，但没有记录

我可以使用 findAndModify 锁来避免双重失败的情况(否则不需要，我们有一个进程级锁)但是我们有过时的锁问题，部分失败仍然可能发生.据我所知(也许是设计?)

是否有针对这种情况的最佳实践?

解决方案

据我所知，没有办法自动删除+保存到不同的集合(也许是设计?)

是的，这是设计使然.MongoDB 明确不提供连接或事务.Remove + Save 是一种交易形式.

<块引用>

是否有针对这种情况的最佳实践?

这里确实有两个低复杂度的选项，都涉及 findAndModify.

选项 1:单个集合

根据您的描述，您基本上是在构建一个具有一些额外功能的队列.如果您利用单个集合，那么您可以使用 findAndModify 来更新正在处理的每个项目的状态.

不幸的是，这意味着您将丢失:...传入"集合可以通过这种方式保持非常小和快速.

选项 2:两个集合

另一个选项基本上是两阶段提交，利用 findAndModify.

查看此此处的文档.>

在 A 中处理一个项目后，您可以设置一个字段来标记它以进行删除.然后将该项目复制到 B.复制到 B 后，您可以从 A 中删除该项目.

I have two collections, one (A) containing items to be processed (relatively small) and one (B) with those already processed (fairly large, with extra result fields).

Items are read from A, get processed and save()'d to B, then remove()'d from A.

The rationale is that indices can be different across these, and that the "incoming" collection can be kept very small and fast this way.

I've run into two issues with this:

• if either remove() or save() time out or otherwise fail under load, I lose the item completely, or process it twice
• if both fail, the side effects happen but there is no record of that

I can sidestep the double-failure case with findAndModify locks (not needed otherwise, we have a process-level lock) but then we have stale lock issues and partial failures can still happen. There's no way to atomically remove+save to different collections, as far as I can tell (maybe by design?)

Is there a Best Practice for this situation?

解决方案

There's no way to atomically remove+save to different collections, as far as I can tell (maybe by design?)

Yes this is by design. MongoDB explicitly does not provides joins or transactions. Remove + Save is a form of transaction.

Is there a Best Practice for this situation?

You really have two low-complexity options here, both involve findAndModify.

Option #1: a single collection

Based on your description, you are basically building a queue with some extra features. If you leverage a single collection then you use findAndModify to update the status of each item as it is processing.

Unfortunately, that means you will lose this: ...that the "incoming" collection can be kept very small and fast this way.

Option #2: two collections

The other option is basically a two phase commit, leveraging findAndModify.

Take a look at the docs for this here.

Once an item is processed in A you set a field to flag it for deletion. You then copy that item over to B. Once copied to B you can then remove the item from A.

这篇关于在集合之间移动文档是表示 MongoDB 中状态变化的好方法吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！