无法通过上传到AWS S3来触发AWS Lambda [英] Unable to trigger AWS Lambda by upload to AWS S3

问题描述

我正在尝试构建一个Kibana仪表板，该仪表板填充有通过AWS Kinesis firehose收集的twitter数据，其中数据传递到S3存储桶中，该S3存储桶触发Lambda函数，该Lambda函数将数据传递到AWS Elastic Search，然后传递到Kibana.我正在关注此博客

I am trying to build a Kibana dashboard fed with twitter data collected via AWS Kinesis firehose where data passes into an S3 bucket which triggers a Lambda function which passes the data to AWS Elastic Search and then to Kibana. I am following this blog https://aws.amazon.com/blogs/big-data/building-a-near-real-time-discovery-platform-with-aws/

数据已正确加载到S3存储桶中，但从未到达Kibana，我相信这是因为Lambda函数并未像我希望的那样被S3中的事件触发(没有调用或日志).我认为这是因为我没有正确设置权限.可以通过测试事件手动调用Lambda函数.

The data is loading into the S3 bucket correctly but it never arrives in Kibana, I believe this is because the Lambda function is not being triggered by events in S3 as I would have hoped (there are no invocations or logs). I think this is because I have not set permissions properly. The Lambda function can be invoked manually by the test event.

在Lambda函数页面上，我选择了一个称为lambda_s3_exec_role的现有角色，该角色具有附加的AWSLambdaExecute策略，但我感觉缺少其他一些针对S3的东西.我无法在创建lambda函数部分的博客中关注此行，因为我无法识别这些选项:

On the Lambda function page I chose an existing role which I called lambda_s3_exec_role which has the AWSLambdaExecute policy attached to it but I feel I'm missing something else more specific to S3. I have been unable to follow this line in the blog in the create lambda function section because I do not recognise those options:

"10.选择lambda_s3_exec_role(如果该值不存在，请选择创建新角色S3执行角色".")

"10. Choose lambda_s3_exec_role (if this value does not exist, choose Create new role S3 execution role)."

有人可以帮助我为Lambda函数创建适当的角色/策略，还是发现问题所在?

从我目前对Lambda函数的查看权限:

From view permissions on the Lambda function I currently have:

功能政策

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "****",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "****",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:s3:::****"
        }
      }
    }
  ]
}

执行角色

{
  "roleName": "lambda_s3_exec_role",
  "policies": [
    {
      "document": {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
          },
          {
            "Effect": "Allow",
            "Action": [
              "s3:GetObject",
              "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::*"
          }
        ]
      },
      "name": "AWSLambdaExecute",
      "id": "****",
      "type": "managed",
      "arn": "arn:aws:iam::aws:policy/AWSLambdaExecute"
    }
  ]
}

推荐答案

您列出的权限看起来不错，因此我将尝试提供一些可能有助于发现问题的步骤，因为很难明确地了解您的问题所在的位置会.

The permissions you have listed look OK so I am going to try provide some steps that might help find the issue as it is difficult to understand specifically where your issue might be.

1. 执行角色是否与 lambda.amazonaws.com
的受信任实体具有信任关系
2. 您的事件前缀是否与firehose中的前缀匹配?在本教程中，它们都是 twitter/raw-data/.如果firehose正在写入不是事件前缀的路径，则不会调用该事件.
3. 当您手动调用它
4. 当您手动调用lambda时是否将其写入日志
5. 使用虚拟数据(以下示例数据)测试lambda

1. Does the execution role have the trust relationship with a trusted entity of lambda.amazonaws.com
2. Does your event prefix match the prefix in firehose. In the tutorial they are both twitter/raw-data/. If firehose is writing to a path that isn't the event prefix then the event won't be invoked.
3. Does the lambda trigger any errors when you manually invoke it
4. Does the lambda write to the logs when you manually invoke it
5. Test the lambda using dummy data (example data below)

CLI

aws lambda invoke \
--invocation-type RequestResponse \
--function-name helloworld \
--region region \
--log-type Tail \
--payload file://dummy_event.json \
--profile adminuser \
outputfile.txt 

示例数据

来源

dummy_event.json

{
   "Records":[  
      {  
         "eventVersion":"2.0",
         "eventSource":"aws:s3",
         "awsRegion":"us-west-2",
         "eventTime":"1970-01-01T00:00:00.000Z",
         "eventName":"ObjectCreated:Put",
         "userIdentity":{  
            "principalId":"AIDAJDPLRKLG7UEXAMPLE"
         },
         "requestParameters":{  
            "sourceIPAddress":"127.0.0.1"
         },
         "responseElements":{  
            "x-amz-request-id":"C3D13FE58DE4C810",
            "x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
         },
         "s3":{  
            "s3SchemaVersion":"1.0",
            "configurationId":"testConfigRule",
            "bucket":{  
               "name":"sourcebucket",
               "ownerIdentity":{  
                  "principalId":"A3NL1KOZZKExample"
               },
               "arn":"arn:aws:s3:::sourcebucket"
            },
            "object":{  
               "key":"HappyFace.jpg",
               "size":1024,
               "eTag":"d41d8cd98f00b204e9800998ecf8427e",
               "versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
            }
         }
      }
   ]
}

这篇关于无法通过上传到AWS S3来触发AWS Lambda的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！