Amazon SQS 长轮询未返回所有消息 [英] Amazon SQS Long Polling not returning all messages

问题描述

我需要在 1 次读取中读取我的 Amazon SQS 队列中的所有消息，然后根据创建的时间戳对其进行排序并对其进行业务逻辑.

I have a requirement to read all messages in my Amazon SQS queue in 1 read and then sort it based on created timestamp and do business logic on it.

为了确保检查所有 SQS 主机的消息，我启用了长轮询.我这样做的方法是将队列的默认等待时间设置为 10 秒.(任何大于 0 的值都将启用长轮询).

To make sure all the SQS hosts are checked for messages, I enabled long polling. The way I did that was to set the default wait time for the queue as 10 seconds. (Any value more than 0 will enable long polling).

然而，当我试图读取队列时，它仍然没有给我所有的消息，我不得不多次读取才能获得所有的消息.我什至通过每个接收请求的代码启用了长轮询，但仍然无效.下面是我正在使用的代码.

However when I tried to read the queue, it still did not give me all the messages and I had to do multiple reads to get all the messages. I even enabled long polling through code per receive request, still did not work. Below is the code I am using.

AmazonSQSClient sqsClient = new AmazonSQSClient(new ClasspathPropertiesFileCredentialsProvider());
sqsClient.setEndpoint("sqs.us-west-1.amazonaws.com");
String queueUrl = "https://sqs.us-west-1.amazonaws.com/12345/queueName";
ReceiveMessageRequest receiveRequest = new ReceiveMessageRequest().withQueueUrl(queueUrl).withMaxNumberOfMessages(10).withWaitTimeSeconds(20);
List<Message> messages = sqsClient.receiveMessage(receiveRequest).getMessages();

我在队列中有 3 条消息，每次运行代码时我都会得到不同的结果，有时我得到所有 3 条消息，有时只有 1 条.我将可见性超时设置为 2 秒，只是为了消除消息变得不可见作为在阅读中没有看到它们的原因.这是短轮询的预期行为.长轮询应该消除多次轮询.我在这里做错了什么吗?

I have 3 messages in the queue and each time I run the code I get a different result, sometimes I get all 3 messages, sometimes just 1. The visibility timeout I set as 2 seconds, just to eliminate the messages becoming invisible as the reason for not seeing them in the read. This is the expected behavior for short polling. Long polling is supposed to eliminate multiple polls. Is there anything I am doing wrong here?

谢谢

推荐答案

长轮询旨在消除多次轮询

Long polling is supposed to eliminate multiple polls

不，当消息实际可用时，长轮询应该消除大量空轮询和虚假空响应.SQS 中的长轮询不会坐等最长的等待时间，只是寻找更多要返回的东西，或者一旦找到东西就继续搜索.SQS 中的长轮询只需要等待足够长的时间才能找到某些东西:

No, long polling is supposed to eliminate a large number of empty polls and false empty responses when messsages are actually available. A long poll in SQS won't sit and wait for the maximum amount of wait time just looking for more things to return, or keep searching once it's found something. A long poll in SQS only waits long enough to find something:

长轮询允许 Amazon SQS 服务在发送响应之前等待队列中的消息可用.因此，除非连接超时，对 ReceiveMessage 请求的响应将包含至少一个可用消息(如果有)和最多 ReceiveMessage 调用中请求的最大数量."

"Long polling allows the Amazon SQS service to wait until a message is available in the queue before sending a response. So unless the connection times out, the response to the ReceiveMessage request will contain at least one of the available messages (if any) and up to the maximum number requested in the ReceiveMessage call."

^{——http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html (强调)}

因此，SQS 找到并返回的某物"可能是所有消息(最多可达您的最大值)，或消息的一个子集，因为如前所述，SQS 是一个分布式系统.一旦我们发现了一些东西，就尽快返回"和返回"之间可能会做出架构决定.以及在整个系统中搜索所有可能的内容，直到客户端将接受的最大消息数"......而且，考虑到这些替代方案，大多数应用程序更喜欢尽快给我你能提供的任何东西"的更快响应似乎是合理的.

So, the "something" that SQS finds and returns may be all of the messages (up to your max), or a subset of the messages, because, as has been mentioned, SQS is a distributed system. There was likely an architectural decision to be made between "return as quickly as possible once we've found something" and "search the entire system for everything possible up to the maximum number of message the client will accept" ... and, given those alternatives, it seems reasonable that most applications would prefer the faster response of "give me whatever you can, as quickly as you can."

直到你从一个长轮询中得到一个空响应，你才知道你实际上已经排空了一个队列.

You don't know that you've actually drained a queue until you get back an empty response from a long poll.

这篇关于Amazon SQS 长轮询未返回所有消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！