带boto3的DynamoDB-限制充当页面大小 [英] DynamoDB with boto3 - limit acts as page size

问题描述

根据boto3 docs ，通过 query 中的 limit 自变量，您可以限制 DynamoDB表/GSI中已评估对象的数量.

According to the boto3 docs, the limit argument in query allows you to to limit the number of evaluated objects in your DynamoDB table/GSI.

但是，当达到所需的限制时，不会返回 LastEvaluatedKey ，因此想要限制获取结果数量的客户端将无法这样做

However, LastEvaluatedKey isn't returned when the desired limit is reached and therefore a client that would like to limit the number of fetched results will fail to do so

考虑以下代码:

        while True:
            query_result = self._dynamodb_client.query(**query_kwargs)
            for dynamodb_formatted_item in query_result["Items"]:
                yield self._convert_dict_from_dynamodb_key_names(
                    from_dynamodb_formatted_dict_to_dict(dynamodb_formatted_item)
                )

            if "LastEvaluatedKey" not in query_result:
                return

我在这里想念什么吗?这是Boto库中的错误吗?

Am I missing something here ? Is this a bug in the Boto library ?

推荐答案

您的示例代码缺少将LastEvaluatedKey作为ExclusiveStartKey参数反馈到查询中的关键部分！因此，您要在循环中重试相同的查询，而不是在上一个查询停止的地方继续进行.

Your example code is missing the critical part where LastEvaluatedKey is fed back into the query, as an ExclusiveStartKey parameter! So you are retrying the same query in a loop, rather than continuing where the previous query stopped.

例如，这是工作代码(我生成了一个数组，它不是像您做的那样酷的生成器;-)):

For example, here is working code (I generated an array, it's not a cool generator like you did ;-)):

def full_query(table, **kwargs):
    response = table.query(**kwargs)
    items = response['Items']
    while 'LastEvaluatedKey' in response:
        response = table.query(ExclusiveStartKey=response['LastEvaluatedKey'], **kwards)
        items.extend(response['Items'])
    return items

您现在可以运行

full_query(Limit=37, KeyConditions={...})

并获取所有结果，分批提取37个.

And get all the results, fetched in batches of 37.

这篇关于带boto3的DynamoDB-限制充当页面大小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！