分页查询/迭代器配方 [英] paginated queries / iterator recipe

问题描述

我经常看到这种模式.

在服务器上:

// Get a bounded number of results, along with a resume token to use 
// for the next call. Successive calls yield a "weakly consistent" view of 
// the underlying set that may or may not reflect concurrent updates.
public<T> String getObjects(
        int maxObjects, String resumeToken, List<T> objectsToReturn);

在客户端上:

// An iterator wrapping repeated calls to getObjects(bufferSize, ...)
public<T> Iterator<T> getIterator(int bufferSize);

大多数地方使用这两种方法的版本，其实现很难实现.有很多边缘案例错误.

Most places roll their own versions of these two methods, and the implementations are surprisingly difficult to get right. There are a lot of edge case bugs.

这些查询是否有规范的配方或库?

Is there a canonical recipe or library for these queries?

(您可以对服务器端存储进行一些简化的假设，例如T具有自然顺序).

(you can make some simplifying assumptions for the server-side storage, e.g. T has a natural ordering).

推荐答案

这里是使用google-guava库中的AbstractIterator和spring-jdbc来实际查询数据库的一种方法:

Here is one using AbstractIterator from google-guava library and spring-jdbc to actually query the database:

public Iterable<T> queryInBatches(
        final String query,
        final Map<String, Integer> paramMap,
        final int pageSize, final Class<T> elementType) {
    return new Iterable<T>() {
        @Override
        public Iterator<T> iterator() {
            final Iterator<List<T>> resultIter = 
                    queryResultIterator(query, paramMap, pageSize, elementType);

            return new AbstractIterator<T>() {
                private Iterator<T> rowSet;

                @Override
                protected T computeNext() {
                    if (rowSet == null) {
                        if (resultIter.hasNext()) {
                            rowSet = resultIter.next().iterator();
                        } else {
                            return endOfData();
                        }
                    }

                    if (rowSet.hasNext()) {
                        return rowSet.next();
                    } else {
                        rowSet = null;
                        return computeNext();
                    }
                }};
        }};
}


private AbstractIterator<List<T>> queryResultIterator(
        final String query, final Map<String, Integer> paramMap, 
        final int pageSize, final Class<T> elementType) {
    return new AbstractIterator<List<T>>() {
        private int page = 0;

        @Override
        protected List<T> computeNext() {
            String sql = String.format(
                    "%s limit %s offset %s", query, pageSize, page++ * pageSize);
            List<T> results = jdbc().queryForList(sql, paramMap, elementType);
            if (!results.isEmpty()) {
                return results;
            } else {
                return endOfData();
            }
        }};
}

AbstractIterator隐藏了大多数涉及编写自己的Iterator实现的复杂性.您只需实现computeNext方法，该方法要么在迭代器中返回下一个值，要么调用endOfData表示迭代器中没有更多的值.

AbstractIterator hides most of the complications involving writing your own implementation of Iterator. You need to only implement the computeNext method which either returns the next value in the iterator or calls endOfData to indicate the there are no more values in the iterator.

这篇关于分页查询/迭代器配方的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！