寻找想法/替代方案来提供匹配 GAE 数据存储查询的项目的页面/项目计数/导航 [英] looking for ideas/alternatives to providing a page/item count/navigation of items matching a GAE datastore query

问题描述

我喜欢数据存储的简单性、可扩展性和易用性；新的 ndb 库中的增强功能非常棒.

I like the datastore simplicity, scalability and ease of use; and the enhancements found in the new ndb library are fabulous.

据我了解数据存储最佳实践，当匹配查询的项目数量很大时，不应编写代码来提供匹配查询结果的项目和/或页数；因为这样做的唯一方法是检索所有资源密集型的结果.

As I understand datastore best practices, one should not write code to provide item and/or page counts of matching query results when the number of items that match a query is large; because the only way to do this is to retrieve all the results which is resource intensive.

但是，在许多应用程序中，包括我们的应用程序，人们普遍希望看到匹配项的计数并为用户提供导航到这些结果的特定页面的能力.如文章 对大型数据集进行分页.为了支持推荐的方法，数据必须包含一个唯一值的列，该列可以按照显示结果的方式进行排序.此列将为每页结果定义一个起始值；保存它，我们可以有效地获取相应的页面，允许根据请求导航到特定页面或下一个页面.因此，如果您想显示以多种方式排序的结果，可能需要维护多个这样的列.

However, in many applications, including ours, it is a common desire to see a count of matching items and provide the user with the ability to navigate to a specific page of those results. The datastore paging issue is further complicated by the requirement to work around limitations of fetch(limit, offset=X) as outlined in the article Paging Through Large Datasets. To support the recommended approach, the data must include a uniquely valued column that can be ordered in the way the results are to be displayed. This column will define a starting value for each page of results; saving it, we can fetch the corresponding page efficiently, allowing navigation to a specific or next page as requested. Therefore, if you want to show results ordered in multiple ways, several such columns may need to be maintained.

需要注意的是，从 SDK v1.3.1 开始，查询游标 是进行数据存储分页的推荐方式. 它们有一些限制，包括缺乏对 IN 和 != 过滤器运算符的支持.目前，我们的一些重要查询使用 IN，但我们将尝试使用 OR 编写它们以与查询游标一起使用.

It should be noted that as of SDK v1.3.1, Query Cursors are the recommended way to do datastore paging. They have some limitations, including lack of support for IN and != filter operators. Currently some of our important queries use IN, but we'll try writing them using OR for use with query cursors.

按照建议的指南，可以为用户提供(下一个) 和(上一个) 导航按钮，以及导航进行时的特定页面按钮.例如，如果用户按 (Next) 3 次，应用程序可以显示以下按钮，记住每个按钮的唯一起始记录或光标以保持导航效率:(Prev) (Page-1) (Page-2) (Page-3) (Page-4) (Next).

Following the guidelines suggested, a user could be given a (Next) and (Prev) navigation buttons, as well as specific page buttons as navigation proceeded. For example if the user pressed (Next) 3 times, the app could show the following buttons, remembering the unique starting record or cursor for each to keep the navigation efficient: (Prev) (Page-1) (Page-2) (Page-3) (Page-4) (Next).

有些人建议单独跟踪计数，但当用户被允许查询一组丰富的字段，而这些字段会改变返回的结果时，这种方法并不实用.

Some have suggested keeping track of counts separately, but this approach isn't practical when users will be allowed to query on a rich set of fields that will vary the results returned.

我正在寻找对这些问题的总体见解，特别是以下问题:

1. 您在数据存储应用中提供哪些查询结果导航选项来解决这些限制?

1. What navigational options of query results do you provide in your datastore apps to work around these limitations?

如果为用户提供高效的结果计数和页面导航整个查询结果集的优先级，应该使用datastore放弃，转而支持现在提供的 GAE MySql 解决方案.

If providing users with efficient result counts and page navigation of the entire query result set is a priority, should use of the datastore be abandoned in favor of the GAE MySql solution now being offered.

大表架构是否有任何即将发生的变化或数据存储实现将提供额外的能力有效地计算查询结果?

Are there any upcoming changes in the big table architecture or datastore implementation that will provide additional capability for counting results of a query efficiently?

非常感谢您的帮助.

推荐答案

这完全取决于您通常获得多少结果.例如.通过传递 .count() 一个合适的限制，你可以提供一个精确的计数，如果 #items 是例如<= 100 和许多"，如果有更多.听起来您无法预先计算所有可能的计数，但至少您可以缓存它们，从而节省许多数据存储操作.

It all depends on how many results you typically get. E.g. by passing .count() a suitable limit you can provide an exact count if the #items is e.g. <= 100 and "many" if there are more. It sounds like you cannot pre-compute all possible counts, but at least you could cache them, thereby saving many datastore ops.

使用 NDB，最有效的方法可能是使用 fetch_page() 请求实体的第一页，然后使用结果游标作为 count() 调用的起点；或者，您最好使用其异步设施同时运行第一页的 fetch() 和 count() .如果您的查询不支持游标，则第二个选项可能是您唯一的选择.大多数 IN/OR 查询当前不支持游标，但如果您按 __key__ 排序，它们会支持.

Using NDB, the most efficient approach may either be to request the first page of entities using fetch_page(), and then using the resulting cursor as a starting point for a count() call; or alternatively, you may be better off running the fetch() of the first page and the count() concurrently using its async facilities. The second option may be your only choice if your query does not support cursors. Most IN / OR queries don't currently support cursors, but they do if you order by __key__.

在UI选项方面，我认为提供下一页和上一页选项就足够了；Gooooooogle"用户界面可以跳过几页很可爱，但我自己几乎从不使用它.(要实现上一页"，请颠倒查询的顺序并使用与当前页面相同的光标.我很确定这可以保证工作.)

In terms of UI options, I think it's sufficient to offer next and previous page options; the "Gooooooogle" UI that affords skipping ahead several pages is cute but I almost never use it myself. (To implement "previous page", reverse the order of the query and use the same cursor you used for the current page. I'm pretty sure this is guaranteed to work.)

这篇关于寻找想法/替代方案来提供匹配 GAE 数据存储查询的项目的页面/项目计数/导航的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！