Lambda仅插入4条记录，但迭代248次 [英] Lambda only inserts 4 records but iterates 248 times

问题描述

我有一个AWS lambda(将由cloudwatch事件触发)，该lambda应该从一个API提取248条记录，并将248条记录插入Dynamo DB.它成功提取了248条记录，但是Dynamo表在执行完成后仅包含4条记录，没有错误.

这是来自lambda的工作马代码:

 公共异步任务< APIGatewayProxyResponse>Get(APIGatewayProxyRequest请求，ILambdaContext上下文){尝试{var count =等待LoadAutocomplete(Data);context.Logger.LogLine($''{{}}个项目已处理到表'{EnvironmentHelper.DynamoTableName}'''));...返回响应；}抓住(前例外){...}}私有异步Task< int>LoadAutocomplete(IList< MatItem>数据){var count = 0;var client = new AmazonDynamoDBClient();var table = Amazon.DynamoDBv2.DocumentModel.Table.LoadTable(client，EnvironmentHelper.DynamoTableName);foreach(数据中的可变项){var doc = new Amazon.DynamoDBv2.DocumentModel.Document();doc.Add("LANGUAGE"，item.LANGUAGE)；doc.Add("MAT_DESC"，item.MAT_DESC)；doc.Add("SALES_ORG"，item.SALES_ORG)；doc.Add("COUNTRY"，item.COUNTRY);doc.Add("SPECIFICATION"，item.SPECIFICATION)；doc.Add("MATERIAL"，item.MATERIAL)；doc.Add("TEMPLATE_ID"，item.TEMPLATE_ID)；等待table.PutItemAsync(doc);数++;System.Threading.Thread.Sleep(100);}返回计数；} 

这是 LogLine 语句的输出:

248个项目已处理到表"my-data"中

以下是用于创建表格的tf:

 #创建表资源"aws_dynamodb_table"；"prospectmaterials_table"{名称=我的表"；hash_key =材料"；billing_mode =已提供"；read_capacity = 5write_capacity = 5属性 {名称=材料"；类型="S"}} 

解决方案

您的代码看起来正确，因此数据和表设计很可能会给您带来困扰.

在DynamoDB中，主键是唯一标识项的键，您需要主键来写入/更新项.您正在将分区键(在本例中也是主键)设置为 MATERIAL 属性.

我的猜测是，数据中 MATERIAL 的值没有很多不同，因此，您的248次写入会导致产生四个项，因为它们可能会覆盖现有项./p>

I have an AWS lambda (to be fired by a cloudwatch event) which should be pulling 248 records from one API and inserting 248 records into Dynamo DB. It successfully pulls 248 records, but the Dynamo table only contains 4 records once the execution finishes, with no errors.

This is the work horse code from the lambda:

public async Task<APIGatewayProxyResponse> Get(APIGatewayProxyRequest request, ILambdaContext context)
{
    try
    {
        var count = await LoadAutocomplete(Data);

        context.Logger.LogLine($"{count} items processed into table '{EnvironmentHelper.DynamoTableName}'");

        ...

        return response;
    }
    catch (Exception ex)
    {
        ...
    }
}

private async Task<int> LoadAutocomplete(IList<MatItem> data)
{
    var count = 0;

    var client = new AmazonDynamoDBClient();
    var table = Amazon.DynamoDBv2.DocumentModel.Table.LoadTable(client, EnvironmentHelper.DynamoTableName);

    foreach (var item in data)
    {
        var doc = new Amazon.DynamoDBv2.DocumentModel.Document();
        doc.Add("LANGUAGE", item.LANGUAGE);
        doc.Add("MAT_DESC", item.MAT_DESC);
        doc.Add("SALES_ORG", item.SALES_ORG);
        doc.Add("COUNTRY", item.COUNTRY);
        doc.Add("SPECIFICATION", item.SPECIFICATION);
        doc.Add("MATERIAL", item.MATERIAL);
        doc.Add("TEMPLATE_ID", item.TEMPLATE_ID);

        await table.PutItemAsync(doc);

        count++;
        System.Threading.Thread.Sleep(100);
    }

    return count;
}

And this is the output of the LogLine statement:

248 items processed into table 'my-data'

Here is the tf used to create the table:

# create the table
resource "aws_dynamodb_table" "prospectmaterials_table" {
  name              = "my-table"
  hash_key          = "MATERIAL"
  billing_mode      = "PROVISIONED"
  read_capacity     = 5
  write_capacity    = 5
  
  attribute {
    name = "MATERIAL"
    type = "S"
  }
}

解决方案

Your code looks right, so it's most likely the data and table design that are going to bite you.

In DynamoDB the primary key is what uniquely identifies an item and you need the primary key to write to/update an item. You're partition key (which is in this case also the primary key) is set to the MATERIAL attribute.

My guess is, that there aren't many distinct values for MATERIAL in the data and as a result of that your 248 writes result in four items, because they probably overwrite the existing items.

这篇关于Lambda仅插入4条记录，但迭代248次的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！