CouchDB复制忽略了零星的文档 [英] CouchDB replication ignoring sporadic documents

问题描述

我为我的应用程序提供了一个CouchDB设置(CouchDB 2.1.1)，它在很大程度上依赖于复制完整性.我们正在使用每个用户一个数据库"的方法，另外还有一层角色"数据库:将用户分组，如下图所示.

最近，在增加Beta测试人员的数量的同时，我们发现某些文档并未得到应有的复制.我们无法看到文档大小，创建/更新时间，用户或其他方面的任何模式.错误似乎偶尔发生，其中2-3个成功复制了文档，然后是4-6个未复制了文档.

服务器在这些文档上以 {"error":"not_found"，"reason":"missing"} 进行响应.

大多数(但不是全部)用户文档已被复制到相应的Role DB，但很少将其一直复制到Master DB.用<进行测试时，从未发生过这种情况.100个文档(现在数据库中有1000-1200个文档).

我发现

解决方案

我过去遇到过类似的情况-当尝试在没有足够权限的情况下复制文档时，复制将失败，这是应该做的.但是，解决了权限问题后，尽管对文档进行编辑/保存解决了该问题，但是您无法复制尝试复制的文档.我想知道这是否归因于检查站? CouchDb手册说到"use_checkpoints"标志:

不建议禁用检查点，因为CouchDB会扫描源数据库的更改提要从头开始.

尽管从头开始进行扫描听起来可能可以解决问题，所以禁用检查点可能会有所帮助.当时我再也没有解决过这个问题，因此恐怕这不是一个正确的答案，只是一个建议.

I've got a CouchDB setup (CouchDB 2.1.1) for my app, which relies heavily on replication integrity. We are using the "one db per user" approach, with an additional layer of "role" db:s that groups users like the image below.

Recently, while increasing the number of beta testers, we discovered that some documents had not been replicated as they should. We are unable to see any pattern in document size, creation/update time, user or other. The errors seem to happen sporadically, with 2-3 successfully replicated docs followed by 4-6 non-replicated docs.

The server responds with {"error":"not_found","reason":"missing"} on those docs.

Most (but not all) of the user documents has been replicated to the corresponding Role DB, but very few made it all the way to the Master DB. This never happened when testing with < 100 documents (now we're at 1000-1200 docs in the db).

I discovered a problem with the "max open files" setting mentioned in the Performance chapter in the docs and fixed it, but the non-replicated documents are still not replicating. If I open a document and save it, it will replicate.

This is my current theory:

1. The replication process tried to copy new documents when the user went online
2. The write process failed due to Linux's "max_open_files" peaked
3. The master DB still thinks the replication was successful
4. At a later replication, the master DB ignores those old documents and only tries to replicate new ones

Could this be correct? And can I somehow make the CouchDB server "double check" all documents and the integrity of previous replications?

Thank you for your time and any helpful comments!

解决方案

I have experienced something similar in the past - when attempting to replicate documents without sufficient permissions the replication fails as it should do. But when the permissions issue is fixed the documents you attempted to replicate cannot then be replicated, although edit/save on the documents fixes the issue. I wonder if this is due to checkpoints? The CouchDb manual says about the "use_checkpoints" flag:

Disabling checkpoints is not recommended as CouchDB will scan the Source database’s changes feed from the beginning.

Though scanning from the beginning sounds like it might fix the problem, so perhaps disabling checkpoints could help. I never got back to that issue at the time so I am afraid this is not a proper answer, just a suggestion.

这篇关于CouchDB复制忽略了零星的文档的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！