MongoDB中的游标是什么? [英] What is a Cursor in MongoDB?

问题描述

我们为某些Morphia查询最终出现cursor not found exceptions而感到困扰. asList ，我发现了关于SO的提示，这可能会消耗大量内存.

We are troubled by eventually occurring cursor not found exceptions for some Morphia Queries asList and I've found a hint on SO, that this might be quite memory consumptive.

现在，我想进一步了解背景知识:sombody可以用英语解释一下，什么是Cursor(在MongoDB中)实际上是什么?为什么它可以保持打开状态或找不到?

Now I'd like to know a bit more about the background: can sombody explain (in English), what a Cursor (in MongoDB) actually is? Why can it kept open or be not found?

文档将光标定义为:

The documentation defines a cursor as:

指向查询结果集的指针.客户可以遍历游标以检索结果.默认情况下，光标闲置10分钟后会超时

A pointer to the result set of a query. Clients can iterate through a cursor to retrieve results. By default, cursors timeout after 10 minutes of inactivity

但这不是很能说明问题.定义查询结果的batch可能会有所帮助，因为文档也说明:

But this is not very telling. Maybe it could be helpful to define a batch for query results, because the documentation also states:

MongoDB服务器分批返回查询结果.批处理大小不会超过BSON文档的最大大小.对于大多数查询，第一批返回101个文档或刚好超过1兆字节的文档.随后的批处理大小为4 MB. [...]对于包含不带索引的排序操作的查询，服务器必须在返回任何结果之前将所有文档加载到内存中以执行排序.

The MongoDB server returns the query results in batches. Batch size will not exceed the maximum BSON document size. For most queries, the first batch returns 101 documents or just enough documents to exceed 1 megabyte. Subsequent batch size is 4 megabytes. [...] For queries that include a sort operation without an index, the server must load all the documents in memory to perform the sort before returning any results.

注意:在所查询的查询中，我们根本不使用排序语句，但也不使用limit和offset.

Note: in our queries in question we don't use sort statements at all, but also no limit and offset.

推荐答案

我绝不是mongodb专家，但我只想补充一下过去一年在中型mongo系统中的一些观察结果.也要感谢@xameeramir出色地介绍了游标的总体工作原理.

I am by no mean a mongodb expert but I just want to add some observations from working in a medium sized mongo system for the last year. Also thanks to @xameeramir for the excellent walkthough about how cursors work in general.

光标丢失"异常的原因可能有多种.我已经注意到的一个在此答案中进行了解释.

The causes of a "cursor lost" exception may be several. One that I have noticed is explained in this answer.

光标位于服务器端.它没有分布在副本集上，而是存在于创建时为主要实例上.这意味着，如果另一个实例接管主要实例，则光标将丢失给客户端.如果旧的主要计算机仍在运行，则它可能仍然存在，但没有用.我猜是过一会儿就把它收拾好了.因此，如果您的mongo副本集不稳定或前面有不稳定的网络，则在执行任何长时间运行的查询时都将不走运.

The cursor lives server side. It is not distributed over a replica set but exists on the instance that is primary at the time of creation. This means that if another instance takes over as primary the cursor will be lost to the client. If the old primary is still up and around it may still be there but for no use. I guess it is garbaged collected away after a while. So if your mongo replica set is unstable or you have a shaky network in front of it you are out of luck when doing any long running queries.

如果游标想要返回的全部内容不适合服务器上的内存，则查询可能会非常慢.服务器上的RAM必须大于您运行的最大查询.

If the full content of what the cursor wants to return does not fit in memory on the server the query may be very slow. RAM on your servers needs to be larger than the largest query you run.

通过更好的设计可以部分避免所有这些.对于长期运行大型查询的用例，最好使用几个较小的数据库集合而不是较大的数据库集合.

All this can partly be avoided by designing better. For a use case with large long running queries you may be better of with several smaller database collections instead of a big one.

这篇关于MongoDB中的游标是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！