S3 put()事件lambda触发器中可以包含多少条记录? [英] How many records can be in S3 put() event lambda trigger?

问题描述

我需要一些lambda函数来实现以下流程:

I need to have some lambda function which got this flow:

触发了S3放置文件事件-> lambda函数->将行插入DynamoDB

Triggered S3 put file event -> lambda function -> insert row to DynamoDB

当我在lambda屏幕上使用AWS创建测试时，在Records列表中只有1条记录的示例如下:

When I'm creating a test using AWS from the lambda screen, I got this example with only 1 record in the Records list:

{
  "Records": [    //  <<<----------------only 1 element 
    {
      "eventVersion": "2.0",
      "eventTime": "1970-01-01T00:00:00.000Z",
      "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
      },
      "s3": {
        "configurationId": "testConfigRule",
        "object": {
          "eTag": "0123456789abcdef0123456789abcdef",
          "sequencer": "0A1B2C3D4E5F678901",
          "key": "HappyFace.jpg",
          "size": 1024
        },
        "bucket": {
          "arn": "arn:aws:s3:::mybucket",
          "name": "roeyg-oregon-s3-bucket",
          "ownerIdentity": {
            "principalId": "EXAMPLE"
          }
        },
        "s3SchemaVersion": "1.0"
      },
      "responseElements": {
        "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH",
        "x-amz-request-id": "EXAMPLE123456789"
      },
      "awsRegion": "us-east-1",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "EXAMPLE"
      },
      "eventSource": "aws:s3"
    }
  ]
}

我尝试了几种方法来查看是否有可能获得一个包含多于一个元素的列表，例如使用CLI或将多个文件一起上传甚至整个文件夹，在所有这些情况下，我每种情况下只有一个一个事件.

I tried in several ways to see if it's possible to get a list with more then one element in this list, like using CLI or upload several files together or even a whole folder, in all of these scenarios I got one item per one event.

我的问题是想知道是否存在一种情况，一次事件我可以获得一个文件以上?

My question is to know if there can be a scenario which I can get more then one file at one event?

那样，我将代码更改为具有循环，而不是建议对AWS的第一个元素进行引用.

By that, I would change my code to have a loop and not a reference to the first element like AWS suggested.

推荐答案

鉴于您当前的配置，每次调用只会得到一条记录.这是因为每个不同的S3放置事件都会以您的AWS Lambda函数作为接收者触发一个不同的S3事件通知.然后，Lambda最多可以处理 100个并发执行，默认情况下，可以根据您的传入事件发生率自动提高的限制.

Given your current configuration, you will only get one record per invocation. This is because each distinct S3 Put Event triggers a distinct S3 event notification with your AWS Lambda function as the recipient. Lambda can then process up to 100 concurrent executions by default, a limit that can be automatically raised depending on your incoming event rate.

AWS Lambda收到Records作为集合，因为它使用S3事件通知将事件发送到lambda，因此使用对可能返回多个记录的事件类型的预期中，该格式作为集合存在.您可以看到S3事件通知类型的完整列表

AWS Lambda receives Records as a collection because it is using S3 Event Notifications to send the event to lambda, and thus using the S3 notification event message structure. In anticipation of event types that may return more than one record, the format exists as a collection. You can see a full list of S3 event notification types here.

这不会影响来自s3:ObjectCreated:Put之类事件的S3通知，因此您可以按原样保留该函数，因为这是Put通知的工作方式-one event -> one notification -> one invokation.

This doesn't impact S3 notifications from events like s3:ObjectCreated:Put, so you're fine to leave the function as-is because that is how Put notifications are intended to work -- one event -> one notification -> one invokation.

也就是说，如果您仍然希望您的代码每次调用能够处理多个记录，那么将其写入以下任何一个记录都不会有任何危害:

That said, if you still want your code to be able to handle multiple Records per invokation, there is no harm in writing it to either:

• 循环处理记录(文档中提供了以下示例:

• process records in a loop (there is an example available in the docs for this: the python example deployment code that AWS provides loops instead of grabbing only the first record). If you go this route, consider adding further logic to filter duplicates if your application can't tolerate them.
• log an error on >1 Records.

这篇关于S3 put()事件lambda触发器中可以包含多少条记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！