如何限制话题的消费率? [英] How to limit the consuming rate from a topic?

问题描述

有没有其他人解决过以下问题?
我有 SNS 主题 充满了来自 S3 的事件，并且有 Lambda 函数 订阅了这个主题，当有数千个事件被放到这个主题时主题，由于超出并发限制，lambda 函数受到限制.
我不想请求增加并发执行的限制，但我会从主题中减少并发消耗，但我没有找到如何操作的信息做吧.谢谢.

Has anyone else solved the following problem?
I have SNS topic filled with events from S3 and there is Lambda function which is subscribed on this topic and when thousand of events are put to this topic, lambda function is throttled because of exceeding the limit of concurrency.
I don't want to request a limit increase for concurrent executions but I would decrease concurrent consuming from the topic, but I didn't find information how to do it. Thanks.

推荐答案

关于 SNS 的几个选项:

A couple of options regarding SNS:

1) SNS 最大接收率

设置 SNS 最大值接收率.这将限制发送给订阅者的 SNS 消息，但如果您有太多消息以至于在处理之前将被丢弃，这可能不是一个很好的选择.来自文档:

Set the SNS Maximum Receive Rate. This will throttle the SNS messages sent to a subscriber, but may not be a great option if you have so many messages that they will be discarded before they can be processed. From the documentation:

您可以设置 Amazon SNS 每秒发送的最大消息数通过设置最大接收速率发送到订阅的端点环境.Amazon SNS 保存等待发送的消息到一个小时.保留超过一个小时的消息将被丢弃.

You can set the maximum number of messages per second that Amazon SNS sends to a subscribed endpoint by setting the Maximum receive rate setting. Amazon SNS holds messages that are awaiting delivery for up to an hour. Messages held for more than an hour are discarded.

如果您一次只收到数千个事件，请将最大接收速率设置为 Lambda 的 默认并发执行限制 '100' 可能值得一试.

If you're only getting thousands of events at a time, setting the Maximum Receive Rate to Lambda's default concurrent execution limit of '100' might be worth a try.

正如@kndrk 所指出的，此限制目前仅适用于 SNS 主题的 HTTP/HTTPS 订阅者.要解决此问题，您可以通过 AWS API Gateway 公开您的 lambda 函数 并将该端点订阅到 SNS 主题，而不是直接订阅 lambda 函数.

As @kndrk notes, this throttling is currently only available for HTTP/HTTPS subscribers to the SNS topic. To work around this, you can expose your lambda function via AWS API Gateway and subscribe that endpoint to the SNS topic, rather than the lambda function directly.

2) 来自 SQS 的流程

订阅 SQS 队列到SNS 主题并处理来自队列的消息，而不是直接来自 sns 主题.SQS 的单次调用 ReceiveMessage 一次只能处理 10 条消息时间，这样你可能更容易节流.

Subscribe an SQS queue to the SNS topic and process messages from the queue, rather than directly from the sns topic. A single invokation of SQS ReceiveMessage can only handle 10 messages at a time, so that may be easier for you to throttle.

另外值得注意的是，您可以直接发布S3 Events到 AWS Lambda.

It is also worth noting that you can publish S3 Events directly to AWS Lambda.

这篇关于如何限制话题的消费率?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！