AWS Lambda SQS触发器限制/限制 [英] AWS Lambda SQS Trigger Throttle/Limit

问题描述

我有一个AWS SQS队列，我将使用Lambda函数触发器进行设置，以便为添加到队列中的每个项目运行Lambda函数，以执行一些处理工作.

I have an AWS SQS queue that I'm going to setup with a Lambda function trigger to run the Lambda function for every item that gets added to the queue to do some processing work.

处理的第一步将是击中API端点，以获取添加到队列中的每个项目的一些数据，然后将其存储在DynamoDB表中.为了控制成本并控制该过程，我想限制在一定时间内调用Lambda函数的次数.

One step of processing is going to be hitting an API endpoint to get some data back for every item added to the queue, then storing that away in an DynamoDB table. In order to manage costs, and stay in control of this process I'm wanting to throttle the amount of times this Lambda function gets called in a certain period of time.

例如，我希望该函数每天最多运行100次，以免淹没DynamoDB表容量或API端点.最好将它限制在一个位置，使其最多只能同时运行5个并发动作，而在再次运行之间会有1秒的延迟.这种类型的控件将使我能够直接将函数映射到DynamoDB表限制，以确保不超出容量，并且遵守所有API速率限制.

For example, I want this function to be run a maximum of 100 times per day, in order to not overwhelm the DynamoDB table capacity or the API endpoint. It would also be nice to throttle it, to where it will only have a maximum of 5 concurrent actions being run at once, with a 1 second delay between running again. This type of control would allow me to directly map the function to the DynamoDB table limits to ensure I'm not going over capacity, and to I comply with any API rate limits.

我研究了 AWS Lambda管理并发.特别是功能级别并发执行限制"部分.但是本节似乎没有解决每天100次的限制，也没有解决运行下一个功能之间的1秒延迟.

I have looked into AWS Lambda Managing Concurrency. Specifically the "Function Level Concurrent Execution Limit" section. But this section doesn't seem to address the 100 times per day limit, or the 1 second delay between running the next function.

我也知道我可以通过限制一次(或每天)放入队列的项目数来限制它.但是由于我打算如何设计该系统，这会增加很多我想避免的复杂性.

I also know that I could just limit it on the other side, by limiting the number of items I put in the queue at a time (or per day). But because of how I'm planning this system, that would add a lot of complexity that I would love to try to avoid.

是否可以使用AWS SQS和Lambda来实现这一目标并限制这些Lambda函数?

Is there a way using AWS SQS and Lambda to achieve this and limit these Lambda functions?

推荐答案

我不认为有一种方法可以将使用SQS作为事件源的Lambda调用次数限制为每天100次.

I don't think there is a way to limit the number of Lambda invocations when using SQS as an event source to 100 a day.

另一种方法是让安排好的事件触发Lambda，该Lambda轮询SQS队列.虽然这可能会增加从SQS到DynamoDB的数据之间的延迟，但是您将对DynamoDB的写入次数和写入速率有更多的控制.

An alternative method would be to have a scheduled event trigger a Lambda, which polls the SQS queue. While this could increase the lag between data going from SQS to DynamoDB, you'd have a lot more control for the number and rate of writes to DynamoDB.

要管理100个计数，您可以在外部保持每日写入计数状态(例如S3)，也可以将100除以计划调用的次数，例如Lambda的10次调用在处理10条消息后停止.无论哪种方式，您都需要确保Lambda不会在处理邮件的过程中超时.

To manage the 100 count, you could either hold the daily write count state externally (e.g. S3) or divide 100 by the number of scheduled invocations, e.g. 10 invocations of a Lambda which stops after processing 10 messages. Either way, you will need to ensure your Lambda does not time out half way through processing a message.

由于一次Lambda调用将对DynamoDB进行多次写入，因此您可以管理每秒进行多少次写入或建立重试.

As a single invocation of Lambda will be making multiple writes to DynamoDB you can manage how many writes are made per sec, or build in retries.

这篇关于AWS Lambda SQS触发器限制/限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！