如何使用官方的mongodb客户端在nodejs中的mongodb中实现基于光标的分页之类的twitter和facebook api? [英] How to implement twitter and facebook api like cursor based pagination in mongodb in nodejs using official mongodb client?

问题描述

我正在尝试实现分页，该操作不应在数据的不同页面中显示重复数据插入或删除.

I am trying to implement pagination which should not show duplicate data in different pages on data insertion or deletion.

使用基于偏移量的方法.mongodb.com/manual/reference/method/cursor.skip/"rel =" nofollow noreferrer>跳过和如何为mongodb实现分页使用官方的mongodb客户端在node.js中运行?

Basic offset-based approach using skip and limit to paginate data is given on this
How to implement pagination for mongodb in node.js using official mongodb client?

基于偏移量的方法有一个很大的缺陷:如果在调用API之间结果列表发生了变化，索引将发生变化并导致某项被返回两次或被跳过而从不返回

Offset-based approach has a big flaw: if the results list has changed between calls to the API, the indices would shift and cause an item to be either returned twice or skipped and never returned

此问题在
上得到了证明 https://www.sitepoint.com/paginating-real-time-data-cursor-based-pagination/

基于时间的分页方法将略胜一筹，因为不再跳过结果.如果您查询第一页，然后删除一个新项目，则不会在第二页中移动结果，一切都很好.但是，这种方法有一个主要缺陷:如果同时创建了多个项目该怎么办?

Time-based pagination approach would be little better because results are no longer skipped. If you query the first page, and then a new item is deleted, it won’t shift the results in your second page and all is fine. However, this approach has a major flaw: what if there is more than one item that was created at the same time?

基于偏移量和基于时间的方法并不是那么完美.所以我需要twitter和facebook api的工作方式，它使用基于游标的方法，使用nodejs和mongodb的官方客户端?

Offset-based and time-based approaches are not so perfect. So I need the way twitter and facebook api work, which uses cursor based approach, using nodejs and official mongodb client?

推荐答案

基于光标的分页可以使用集合中任何唯一，可排序和不可变的字段来实现.

Cursor based pagination can be implemented using any field in collection which is Unique, Orderable and Immutable.

_id 满足所有唯一，可排序和不可变条件.基于此字段，我们可以排序并返回页面结果，最后一个文档的 _id 作为后续请求的目录.

_id satisfy all Unique, Orderable and Immutable conditions. Based on this field we can sort and return page result with _id of last document as the cusror for subsequent request.

卷曲https://api.mixmax.com/items?limit=2

const items = db.items.find({}).sort({
   _id: -1
}).limit(2);

const next = items[items.length - 1]._id
res.json({ items, next })

当用户想要获得第二页时，他们在URL上传递光标(下一个): curl https://api.mixmax.com/items?limit=2&next=590e9abd4abbf1165862d342

when the user wants to get the second page, they pass the cursor (as next) on the URL: curl https://api.mixmax.com/items?limit=2&next=590e9abd4abbf1165862d342

const items = db.items.find({
  _id: { $lt: req.query.next }
}).sort({
   _id: -1
}).limit(2);

const next = items[items.length - 1]._id
res.json({ items, next })

如果我们要以不同的顺序返回结果(例如项目的日期)，则将 sort = launchDate 添加到查询字符串中. curl https://api.mixmax.com/items?limit=2&sort=launchDate

If we want to return results in a different order, such as the date the item then we will add sort=launchDate to the querystring. curl https://api.mixmax.com/items?limit=2&sort=launchDate

const items = db.items.find({}).sort({
   launchDate: -1
}).limit(2);

const next = items[items.length - 1].launchDate;
res.json({ items, next })

用于后续页面请求
curl https://api.mixmax.com/items?limit=2&sort=launchDate&next=2017-09-11T00%3A44%3A54.036Z

const items = db.items.find({
  launchDate: { $lt: req.query.next }
}).sort({
   _id: -1
}).limit(2);

const next = items[items.length - 1].launchDate;
res.json({ items, next });

如果我们在同一天和同一时间推出了一堆物品?现在，我们的 launchDate 字段不再唯一，并且不满足唯一，可排序和不可变.状况.我们不能将其用作光标字段.但是我们可以使用两个字段来生成游标.由于我们知道MongoDB中的 _id 字段始终满足上述三个条件，因此我们知道如果将其与我们的 launchDate 字段，这两个字段的组合将满足要求，并且可以一起用作光标字段. curl https://api.mixmax.com/items?limit=2&sort=launchDate

If we launched a bunch of items on the same day and time? Now our launchDate field is no longer unique and doesn’t satisfy Unique, Orderable and Immutable. condition. We can’t use it as a cursor field. But we could use two fields to generate the cursor.Since we know that the _id field in MongoDB always satisfies the above three condition, we know that if we use it alongside our launchDate field, the combination of the two fields would satisfy the requirements and could be together used as a cursor field. curl https://api.mixmax.com/items?limit=2&sort=launchDate

const items = db.items.find({}).sort({
   launchDate: -1,
  _id: -1 // secondary sort in case there are duplicate launchDate values
}).limit(2);

const lastItem = items[items.length - 1];
// The cursor is a concatenation of the two cursor fields, since both are needed to satisfy the requirements of being a cursor field
const next = `${lastItem.launchDate}_${lastItem._id}`;
res.json({ items, next });

用于后续页面请求
curl https://api.mixmax.com/items?limit=2&sort=launchDate&next=2017-09-11T00%3A44%3A54.036Z_590e9abd4abbf1165862d342

const [nextLaunchDate, nextId] = req.query.next.split(‘_’);
const items = db.items.find({
  $or: [{
    launchDate: { $lt: nextLaunchDate }
  }, {
    // If the launchDate is an exact match, we need a tiebreaker, so we use the _id field from the cursor.
    launchDate: nextLaunchDate,
  _id: { $lt: nextId }
  }]
}).sort({
   _id: -1
}).limit(2);

const lastItem = items[items.length - 1];
// The cursor is a concatenation of the two cursor fields, since both are needed to satisfy the requirements of being a cursor field
const next = `${lastItem.launchDate}_${lastItem._id}`;
res.json({ items, next });

参考: https://engineering.mixmax.com/blog/api-paging-built-the-right-way/

这篇关于如何使用官方的mongodb客户端在nodejs中的mongodb中实现基于光标的分页之类的twitter和facebook api?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！