AWS:多个实例读取 SQS [英] AWS: multiple instances reading SQS

问题描述

简单的问题:我想在 Amazon 上运行一个自动缩放组，它会启动多个实例来处理来自 SQS 队列的消息.但是我怎么知道这些实例没有处理相同的消息呢?

Simple question: I want to run an autoscale group on Amazon, which fires up multiple instance which processes the messages from a SQS queue. But how do I know that the instances aren't processing the same messages?

我可以在处理消息时从队列中删除它.但是如果它还没有被删除并且仍在被一个实例处理，我认为另一个实例可以下载相同的消息并对其进行处理.

I can delete a message from the queue when it's processed. But if it's not deleted yet and still being processed by an instance, another instance CAN download that same message and processing it also, to my opinion.

推荐答案

除了 SQS 多次错误地传递相同消息的可能性很小(您仍然需要考虑，即使不太可能)，我怀疑您的问题源于对 SQS 的可见性超时"概念不熟悉.

Aside from the fairly remote possibility of SQS incorrectly delivering the same message more than once (which you still need to account for, even though it is unlikely), I suspect your question stems from a lack of familiarity with SQS's concept of "visibility timeout."

组件收到消息后，消息还在队列中.但是，您不希望系统中的其他组件再次接收和处理消息.因此，Amazon SQS 会通过可见性超时阻止它们，在此期间 Amazon SQS 会阻止其他使用组件接收和处理该消息.

Immediately after the component receives the message, the message is still in the queue. However, you don't want other components in the system receiving and processing the message again. Therefore, Amazon SQS blocks them with a visibility timeout, which is a period of time during which Amazon SQS prevents other consuming components from receiving and processing that message.

http://docs.aws.amazon.com/AWSSimpleQueueService/最新/SQSDeveloperGuide/AboutVT.html

这就是阻止多个队列运行者看到相同消息的原因.一旦可见性超时到期，消息将再次传递给队列使用者，除非您将其删除，或者它超过了最大配置的传递次数(此时它会被删除或进入单独的死信队列，如果您已配置)).如果作业所需的时间超过配置的可见性超时，您的使用者还可以向 SQS 发送请求以更改该单个消息的可见性超时.

This is what keeps multiple queue runners from seeing the same message. Once the visibility timeout expires, the message will be delivered again to a queue consumer, unless you delete it, or it exceeds the maximum configured number of deliveries (at which point it's deleted or goes into a separate dead letter queue if you have configured one). If a job will take longer than the configured visibility timeout, your consumer can also send a request to SQS to change the visibility timeout for that individual message.

更新:

由于最初编写此答案，SQS 在某些 AWS 区域中引入了 FIFO 队列.它们以上述相同的逻辑运行，但有保证的按顺序传递和额外的保护措施，以确保不会偶尔发生重复的消息传递.

Since this answer was originally written, SQS has introduced FIFO Queues in some of the AWS regions. These operate with the same logic described above, but with guaranteed in-order delivery and additional safeguards to guarantee that occasional duplicate message delivery cannot occur.

FIFO(先进先出)队列旨在在操作和事件的顺序至关重要或不能容忍重复的情况下增强应用程序之间的消息传递.FIFO 队列还提供仅一次处理，但限制为每秒 300 个事务 (TPS).

FIFO (First-In-First-Out) queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can't be tolerated. FIFO queues also provide exactly-once processing but are limited to 300 transactions per second (TPS).

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html

将应用程序切换到 FIFO 队列确实需要更改一些代码，并且需要创建一个新队列——现有队列不能更改为 FIFO.

Switching an application to a FIFO queue does require some code changes, and requires that a new queue be created -- existing queues can't be changed over to FIFO.

这篇关于AWS:多个实例读取 SQS的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！