python DynamoDB扫描操作未返回所有记录 [英] python DynamoDB scan operation not return all records

问题描述

在DynamoDB表中，当我应用任何过滤器时，我有 161712条记录，我仅收到扫描计数值10589

In DynamoDB table, i have 161712 records when apply with out any filters,i have received scan count value only 10589

这是mytable元

{
  "AttributeDefinitions": [
    {
      "AttributeName": "question_id",
      "AttributeType": "N"
    },
    {
      "AttributeName": "timestamp",
      "AttributeType": "S"
    }
  ],
  "TableName": "users_answers",
  "KeySchema": [
    {
      "AttributeName": "timestamp",
      "KeyType": "HASH"
    },
    {
      "AttributeName": "question_id",
      "KeyType": "RANGE"
    }
  ],
  "TableStatus": "ACTIVE",
  "CreationDateTime": "2017-09-12T12:33:22.615Z",
  "ProvisionedThroughput": {
    "LastIncreaseDateTime": "2017-09-12T16:46:26.742Z",
    "NumberOfDecreasesToday": 0,
    "ReadCapacityUnits": 80,
    "WriteCapacityUnits": 80
  },
  "TableSizeBytes": 16014441,
  "ItemCount": 161712
}

当我在表上方进行扫描操作时，只会得到10589条记录

when i do scan operation above table will get only 10589 records

table = dynamo.get_table('answer_options')
x    = table.scan()

请建议我如何获取表中的全部记录

Please suggest how i fetch entire records from table

Env：python 3.5.1，flask dynamodb

Env: python 3.5.1 , flask dynamodb

预先感谢

推荐答案

DynamoDB每个请求仅返回1MB。您必须遍历并发出多个请求，直到获得整个数据集。

DynamoDB only returns 1MB per request. You have to loop through and make multiple requests until you get your entire dataset.

来自 DynamoDB文档：

DynamoDB分页扫描操作的结果。通过分页，将扫描结果分为大小为1 MB（或更小）的数据页面。应用程序可以处理结果的第一页，然后处理第二页，依此类推。

DynamoDB paginates the results from Scan operations. With pagination, the Scan results are divided into "pages" of data that are 1 MB in size (or less). An application can process the first page of results, then the second page, and so on.

单次扫描将仅返回适合1 MB大小的结果集限制。为了确定是否还有更多结果，并一次检索一页，应用程序应该执行以下操作：

A single Scan will only return a result set that fits within the 1 MB size limit. To determine whether there are more results, and to retrieve them one page at a time, applications should do the following:

1. 检查低级扫描结果：

1. Examine the low-level Scan result:

• 如果结果包含LastEvaluatedKey元素，请继续执行步骤2。


• 如果结果中没有LastEvaluatedKey，则不再有要检索的项目。

构造一个新的Scan请求，其参数与上一个参数相同，但是这次，从步骤1中获取LastEvaluatedKey值，并将其用作新Scan请求中的ExclusiveStartKey参数。

Construct a new Scan request, with the same parameters as the previous one—but this time, take the LastEvaluatedKey value from step 1 and use it as the ExclusiveStartKey parameter in the new Scan request.

运行新的扫描请求。

转到步骤1。

这篇关于python DynamoDB扫描操作未返回所有记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！