AWS:读取SQS的多个实例 [英] AWS: multiple instances reading SQS

问题描述

一个简单的问题: 我想在Amazon上运行一个自动扩展组，该组将启动多个实例，该实例处理来自SQS队列的消息.但是我怎么知道实例没有处理相同的消息?

Simple question: I want to run an autoscale group on Amazon, which fires up multiple instance which processes the messages from a SQS queue. But how do I know that the instances aren't processing the same messages?

处理完毕后，我可以从队列中删除消息.但是，如果我尚未删除它，并且仍由一个实例处理它，那么另一个实例可以下载相同的消息并对其进行处理.

I can delete a message from the queue when it's processed. But if it's not deleted yet and still being processed by an instance, another instance CAN download that same message and processing it also, to my opinion.

推荐答案

除了SQS多次错误地传递同一条消息的可能性非常遥远之外(即使不太可能，您仍然需要考虑这一点)，怀疑您的问题源于对SQS的可见性超时"概念缺乏了解.

Aside from the fairly remote possibility of SQS incorrectly delivering the same message more than once (which you still need to account for, even though it is unlikely), I suspect your question stems from a lack of familiarity with SQS's concept of "visibility timeout."

在组件收到消息后，消息仍在队列中.但是，您不希望系统中的其他组件再次接收和处理消息.因此，Amazon SQS使用可见性超时来阻止它们，在这段时间内，Amazon SQS会阻止其他消耗组件接收和处理该消息.

Immediately after the component receives the message, the message is still in the queue. However, you don't want other components in the system receiving and processing the message again. Therefore, Amazon SQS blocks them with a visibility timeout, which is a period of time during which Amazon SQS prevents other consuming components from receiving and processing that message.

http://docs.aws.amazon.com/AWSSimpleQueueService/最新/SQSDeveloperGuide/AboutVT.html

这是使多个队列运行器看不到相同消息的原因.可见性超时到期后，该消息将再次传递给队列使用者，除非您将其删除，否则它将超过配置的最大传递数量(此时，该消息将被删除，或者如果已配置，则进入单独的死信队列).如果作业花费的时间比配置的可见性超时长，那么您的使用者还可以向SQS发送请求以更改该单个消息的可见性超时.

This is what keeps multiple queue runners from seeing the same message. Once the visibility timeout expires, the message will be delivered again to a queue consumer, unless you delete it, or it exceeds the maximum configured number of deliveries (at which point it's deleted or goes into a separate dead letter queue if you have configured one). If a job will take longer than the configured visibility timeout, your consumer can also send a request to SQS to change the visibility timeout for that individual message.

更新:

自从最初编写此答案以来，SQS在某些AWS区域中引入了FIFO队列.它们的操作与上述逻辑相同，但有保证的按顺序传递和附加的安全措施可确保不会出现偶然的重复消息传递.

Since this answer was originally written, SQS has introduced FIFO Queues in some of the AWS regions. These operate with the same logic described above, but with guaranteed in-order delivery and additional safeguards to guarantee that occasional duplicate message delivery cannot occur.

FIFO(先进先出)队列旨在在操作和事件的顺序很关键或不允许重复的情况下增强应用程序之间的消息传递. FIFO队列还提供一次精确的处理，但每秒限制为300个事务(TPS).

FIFO (First-In-First-Out) queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can't be tolerated. FIFO queues also provide exactly-once processing but are limited to 300 transactions per second (TPS).

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html

将应用程序切换到FIFO队列确实需要进行一些代码更改，并且需要创建一个新队列-无法将现有队列切换到FIFO.

Switching an application to a FIFO queue does require some code changes, and requires that a new queue be created -- existing queues can't be changed over to FIFO.

这篇关于AWS:读取SQS的多个实例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！