使用 CloudFormation 在 S3 存储桶中创建 Lambda 通知 [英] Create a Lambda notification in an S3 bucket with CloudFormation

问题描述

我正在尝试为 CloudFormation 模板中的 Lambda 函数创建 S3 触发器.S3 存储桶已存在，正在创建 Lambda 函数.

描述:将对象上传到 S3 存储桶，触发 Lambda 事件，将对象键作为堆栈输出返回.参数:钥匙:描述:S3 对象键类型:字符串默认值:测试身体:描述:S3 对象主体内容类型:字符串默认:测试内容桶名:说明:S3 存储桶名称(必须已存在)类型:字符串资源:桶配置:类型:自定义::S3BucketConfiguration取决于:- 桶权限- 通知桶策略特性:ServiceToken: !GetAtt S3BucketConfiguration.Arn存储桶: !Ref 存储桶名称通知配置:Lambda 函数配置:- 事件:['s3:ObjectCreated:*']LambdaFunctionArn: !GetAtt BucketWatcher.ArnS3Bucket配置:类型:AWS::Lambda::Function特性:描述:S3 对象自定义资源处理程序:index.handler角色:!GetAtt LambdaExecutionRole.Arn代码:压缩文件:!Sub |var response = require('cfn-response');var AWS = require('aws-sdk');var s3 = 新的 AWS.S3();export.handler = 函数(事件，上下文){var 响应 = (e) =>response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});process.on('uncaughtException', e=>failed(e));var params = event.ResourceProperties;删除 params.ServiceToken;if (event.RequestType === '删除') {params.NotificationConfiguration = {};s3.putBucketNotificationConfiguration(params).promise().then((data)=>respond()).catch((e)=>respond());} 别的 {s3.putBucketNotificationConfiguration(params).promise().then((data)=>respond()).catch((e)=>respond(e));}};超时:30运行时:nodejs4.3桶权限:类型:AWS::Lambda::Permission特性:操作:'lambda:InvokeFunction'函数名称:!Ref BucketWatcher校长:s3.amazonaws.comSourceAccount: !Ref "AWS::AccountId"SourceArn: !Sub "arn:aws:s3:::${BucketName}"桶观察者:类型:AWS::Lambda::Function特性:说明:调用时向 Handle 发送等待条件信号处理程序:index.handler角色:!GetAtt LambdaExecutionRole.Arn代码:压缩文件:!Sub |export.handler = 函数(事件，上下文){console.log("收到请求:
", JSON.stringify(event));var responseBody = JSON.stringify({状态":成功"，"UniqueId" : "密钥",数据":event.Records[0].s3.object.key，原因" : ""});var https = require("https");var url = require("url");var parsedUrl = url.parse('${Handle}');变量选项 = {主机名:parsedUrl.hostname，端口:443，路径:parsedUrl.path，方法:PUT"，标题:{内容类型": ""，内容长度":responseBody.length}};var request = https.request(options, function(response) {console.log("状态码:" + response.statusCode);console.log("状态信息:" + response.statusMessage);上下文.done();});request.on(错误"，函数(错误){console.log("send(..) 执行 https.request(..) 失败:" + error);上下文.done();});request.write(responseBody);request.end();};超时:30运行时:nodejs4.3处理:类型:AWS::CloudFormation::WaitConditionHandle等待:类型:AWS::CloudFormation::WaitCondition特性:句柄: !Ref 句柄超时:300S3对象:类型:自定义::S3Object取决于:BucketConfiguration特性:ServiceToken: !GetAtt S3ObjectFunction.Arn存储桶: !Ref 存储桶名称键: !Ref 键正文: !Ref 正文S3ObjectFunction:类型:AWS::Lambda::Function特性:描述:S3 对象自定义资源处理程序:index.handler角色:!GetAtt LambdaExecutionRole.Arn代码:压缩文件:!Sub |var response = require('cfn-response');var AWS = require('aws-sdk');var s3 = 新的 AWS.S3();export.handler = 函数(事件，上下文){var 响应 = (e) =>response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});var params = event.ResourceProperties;删除 params.ServiceToken;if (event.RequestType == 'Create' || event.RequestType == 'Update') {s3.putObject(params).promise().then((data)=>respond()).catch((e)=>respond(e));} else if (event.RequestType == 'Delete') {删除 params.Body;s3.deleteObject(params).promise().then((data)=>respond()).catch((e)=>respond(e));} 别的 {响应({错误:'无效的请求类型'})；}};超时:30运行时:nodejs4.3LambdaExecutionRole:类型:AWS::IAM::角色特性:AssumeRolePolicyDocument:版本:'2012-10-17'陈述:- 效果:允许委托人:{服务:[lambda.amazonaws.com]}行动:['sts:AssumeRole']小路:/ManagedPolicyArns:- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"政策:- 策略名称:S3Policy政策文件:版本:'2012-10-17'陈述:- 效果:允许行动:- 's3:PutObject'- 'S3:删除对象'资源:!Sub "arn:aws:s3:::${BucketName}/${Key}"通知桶策略:类型:AWS::S3::BucketPolicy特性:存储桶: !Ref 存储桶名称政策文件:陈述:- 效果:允许"行动:- 's3:PutBucketNotification'资源:!Sub "arn:aws:s3:::${BucketName}"主要的:AWS:!GetAtt LambdaExecutionRole.Arn输出:结果:值:!GetAtt Wait.Data

I'm trying to create an S3 trigger for a Lambda function in a CloudFormation Template. The S3 bucket already exists, and the Lambda function is being created.

This says it's not possible to modify pre-existing infrastructure (S3 in this case) with a CFT, but this seems to say that the bucket has to be pre-existing.

1. It seems that the trigger can't be created using a CFT type "AWS::Lambda..." and that the source service needs to create the trigger. In my case, that's a NotificationConfiguration-LambdaConfiguration for an s3 bucket. Is all of that correct?

2. When I try to add a NotificationConfiguration to an existing S3 bucket with a CFT, it says that I can't. Is there any way to do this?

解决方案

Unfortunately, the official AWS::CloudFormation template only allows you to control Amazon S3 NotificationConfiguration as a NotificationConfiguration property of the parent AWS::S3::Bucket Resource, which means that you can't attach this configuration to any existing bucket, you have to apply it to a CloudFormation-managed bucket for it to work.

A workaround is to implement the PUT Bucket Notification API call directly as a Lambda-backed Custom Resource using the putBucketNotificationConfiguration JavaScript API call. However, because modifying the NotificationConfiguration on S3 buckets is restricted to the bucket's creator, you also need to add an AWS::S3::BucketPolicy Resource granting your Lambda Function access to the s3:PutBucketNotification action.

Here's a complete, self-contained CloudFormation template that demonstrates how to trigger a Lambda function whenever a file is added to an existing S3 bucket, using 2 Lambda-Backed Custom Resources (BucketConfiguration to set the bucket notification configuration, S3Object to upload an object to the bucket) and a third Lambda function (BucketWatcher to trigger the Wait Condition when an object is uploaded to the bucket).

Description: Upload an object to an S3 bucket, triggering a Lambda event, returning the object key as a Stack Output.
Parameters:
  Key:
    Description: S3 Object key
    Type: String
    Default: test
  Body:
    Description: S3 Object body content
    Type: String
    Default: TEST CONTENT
  BucketName:
    Description: S3 Bucket name (must already exist)
    Type: String
Resources:
  BucketConfiguration:
    Type: Custom::S3BucketConfiguration
    DependsOn:
    - BucketPermission
    - NotificationBucketPolicy
    Properties:
      ServiceToken: !GetAtt S3BucketConfiguration.Arn
      Bucket: !Ref BucketName
      NotificationConfiguration:
        LambdaFunctionConfigurations:
        - Events: ['s3:ObjectCreated:*']
          LambdaFunctionArn: !GetAtt BucketWatcher.Arn
  S3BucketConfiguration:
    Type: AWS::Lambda::Function
    Properties:
      Description: S3 Object Custom Resource
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          var response = require('cfn-response');
          var AWS = require('aws-sdk');
          var s3 = new AWS.S3();
          exports.handler = function(event, context) {
            var respond = (e) => response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});
            process.on('uncaughtException', e=>failed(e));
            var params = event.ResourceProperties;
            delete params.ServiceToken;
            if (event.RequestType === 'Delete') {
              params.NotificationConfiguration = {};
              s3.putBucketNotificationConfiguration(params).promise()
                .then((data)=>respond())
                .catch((e)=>respond());
            } else {
              s3.putBucketNotificationConfiguration(params).promise()
                .then((data)=>respond())
                .catch((e)=>respond(e));
            }
          };
      Timeout: 30
      Runtime: nodejs4.3
  BucketPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName: !Ref BucketWatcher
      Principal: s3.amazonaws.com
      SourceAccount: !Ref "AWS::AccountId"
      SourceArn: !Sub "arn:aws:s3:::${BucketName}"
  BucketWatcher:
    Type: AWS::Lambda::Function
    Properties:
      Description: Sends a Wait Condition signal to Handle when invoked
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          exports.handler = function(event, context) {
            console.log("Request received:
", JSON.stringify(event));
            var responseBody = JSON.stringify({
              "Status" : "SUCCESS",
              "UniqueId" : "Key",
              "Data" : event.Records[0].s3.object.key,
              "Reason" : ""
            });
            var https = require("https");
            var url = require("url");
            var parsedUrl = url.parse('${Handle}');
            var options = {
                hostname: parsedUrl.hostname,
                port: 443,
                path: parsedUrl.path,
                method: "PUT",
                headers: {
                    "content-type": "",
                    "content-length": responseBody.length
                }
            };
            var request = https.request(options, function(response) {
                console.log("Status code: " + response.statusCode);
                console.log("Status message: " + response.statusMessage);
                context.done();
            });
            request.on("error", function(error) {
                console.log("send(..) failed executing https.request(..): " + error);
                context.done();
            });
            request.write(responseBody);
            request.end();
          };
      Timeout: 30
      Runtime: nodejs4.3
  Handle:
    Type: AWS::CloudFormation::WaitConditionHandle
  Wait:
    Type: AWS::CloudFormation::WaitCondition
    Properties:
      Handle: !Ref Handle
      Timeout: 300
  S3Object:
    Type: Custom::S3Object
    DependsOn: BucketConfiguration
    Properties:
      ServiceToken: !GetAtt S3ObjectFunction.Arn
      Bucket: !Ref BucketName
      Key: !Ref Key
      Body: !Ref Body
  S3ObjectFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: S3 Object Custom Resource
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          var response = require('cfn-response');
          var AWS = require('aws-sdk');
          var s3 = new AWS.S3();
          exports.handler = function(event, context) {
            var respond = (e) => response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});
            var params = event.ResourceProperties;
            delete params.ServiceToken;
            if (event.RequestType == 'Create' || event.RequestType == 'Update') {
              s3.putObject(params).promise()
                .then((data)=>respond())
                .catch((e)=>respond(e));
            } else if (event.RequestType == 'Delete') {
              delete params.Body;
              s3.deleteObject(params).promise()
                .then((data)=>respond())
                .catch((e)=>respond(e));
            } else {
              respond({Error: 'Invalid request type'});
            }
          };
      Timeout: 30
      Runtime: nodejs4.3
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal: {Service: [lambda.amazonaws.com]}
          Action: ['sts:AssumeRole']
      Path: /
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
      Policies:
      - PolicyName: S3Policy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - 's3:PutObject'
                - 'S3:DeleteObject'
              Resource: !Sub "arn:aws:s3:::${BucketName}/${Key}"
  NotificationBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref BucketName
      PolicyDocument:
        Statement:
          - Effect: "Allow"
            Action:
            - 's3:PutBucketNotification'
            Resource: !Sub "arn:aws:s3:::${BucketName}"
            Principal:
              AWS: !GetAtt LambdaExecutionRole.Arn
Outputs:
  Result:
    Value: !GetAtt Wait.Data

这篇关于使用 CloudFormation 在 S3 存储桶中创建 Lambda 通知的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！