为什么 SqS 消息有时会在队列中保持飞行状态 [英] Why do SqS messages sometimes remain in-flight on queue

问题描述

我正在以一种非常简单的方式使用 Amazon SQS 队列.通常，消息被写入并立即可见和阅读.有时，会写入一条消息，并在队列中保持飞行中(不可见)几分钟.我可以从控制台看到它.接收消息等待时间为 0，默认可见性为 5 秒.它将保持这种状态几分钟，或者直到写入新消息以某种方式释放它.延迟几秒是可以的，但是超过60秒就不行了.

有 8 个读取器线程总是长时间轮询，所以并不是某些东西没有尝试读取它，而是.

编辑:需要明确的是，任何消费者读取都不会返回任何消息，无论控制台是否打开都会发生.在这种情况下，只涉及一条消息，它只是坐在队列中，消费者不可见.

有没有其他人看到过这种行为以及我可以做些什么来改进它?

这是我正在使用的 Java sdk:

<依赖><groupId>com.amazonaws</groupId><artifactId>aws-java-sdk</artifactId><version>1.5.2</version></依赖>

这是读取的代码(max=10,maxwait=0 启动配置):

void read(MessageConsumer consumer) {列表<消息>消息 = 读取(最大，最大等待)；对于(消息消息:消息){如果(tryConsume(消费者，消息)){删除(message.getReceiptHandle());}}}私人列表<消息>读(int max，int maxWait){AmazonSQS sqs = getClient();ReceiveMessageRequest rq = 新的 ReceiveMessageRequest(queueUrl);rq.setMaxNumberOfMessages(max);rq.setWaitTimeSeconds(maxWait);列表<消息>消息 = sqs.receiveMessage(rq).getMessages();如果(消息.大小()> 0){LOG.info("从 SQS 队列中读取 {} 条消息",messages.size());}回消息；}

发生这种情况时，read .."的日志行永远不会出现，这也是导致我进入控制台并查看消息是否存在的原因，确实如此.

解决方案

这听起来像是你误解了你所看到的.

消息飞行中"不是待发送的消息，它们是已经发送但未被消费者进一步处理的消息.

<块引用>

如果消息已发送给客户端但尚未删除或尚未到达其可见性窗口的末尾，则认为它们处于传输状态.

^{—http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/sqs-metricscollected.html}

当消费者收到一条消息时，它必须——在某个时候——要么删除该消息，要么向 增加该消息的超时时间；否则在超时到期后消息将再次可见.如果消费者未能执行其中一项操作，该消息将自动再次可见.可见性超时是消费者在必须完成其中一项工作之前的时间.

消息不应在飞行中"；没有已经收到它们的东西——但是那个东西"可以包括控制台本身，正如您在选择查看/删除消息"时看到的弹出窗口中所注意到的那样在控制台中(除非您已经选中了不再显示"复选框):

<块引用>

在控制台停止轮询消息之前，其他应用程序将无法使用控制台中显示的消息.

控制台中显示的消息是在飞行中"；当控制台从查看/删除消息"观察队列时屏幕.

没有明显意义的部分是消息持续了几分钟".如果您的默认可见性超时只有 5 秒，并且您的代码中没有任何内容增加该超时……但是……您的消费者没有正确处理消息，导致它超时并立即重新发送，这几乎可以完美地解释，给人的印象是该消息的单个实例仍在传输中，而实际上，该消息正在短暂地转换回可见状态，但几乎立即被另一个消费者声明，并再次将其带回传输中.

I'm using Amazon SQS queues in a very simple way. Usually, messages are written and immediately visible and read. Occasionally, a message is written, and remains In-Flight(Not Visible) on the queue for several minutes. I can see it from the console. Receive-message-wait time is 0, and Default Visibility is 5 seconds. It will remain that way for several minutes, or until a new message gets written that somehow releases it. A few seconds delay is ok, but more than 60 seconds is not ok.

There a 8 reader threads that are long polling always, so its not that something is not trying to read it, they are.

Edit : To be clear, none of the consumer reads are returning any messages at all and it happens regardless of whether or not the console is open. In this scenario, only one message is involved, and it is just sitting in the queue invisible to the consumers.

Has anyone else seen this behavior and what I can do to improve it?

Here is the sdk for java I am using:

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk</artifactId>
  <version>1.5.2</version>
</dependency>     

Here is the code that does the reading (max=10,maxwait=0 startup config):

void read(MessageConsumer consumer) {

  List<Message> messages = read(max, maxWait);

  for (Message message : messages) {
    if (tryConsume(consumer, message)) {
      delete(message.getReceiptHandle());
    }
  }
}

private List<Message> read(int max, int maxWait) {

  AmazonSQS sqs = getClient();
  ReceiveMessageRequest rq = new ReceiveMessageRequest(queueUrl);
  rq.setMaxNumberOfMessages(max);
  rq.setWaitTimeSeconds(maxWait);
  List<Message> messages = sqs.receiveMessage(rq).getMessages();

  if (messages.size() > 0) {
    LOG.info("read {} messages from SQS queue",messages.size());
  }

  return messages;
}

The log line for "read .." never appears when this is happening, and its what causes me to go in with the console and see if the message is there or not, and it is.

解决方案

It sounds like you are misinterpreting what you are seeing.

Messages "in flight" are not pending delivery, they're messages that have already been delivered but not further acted on by the consumer.

Messages are considered in flight if they have been sent to a client but have not yet been deleted or have not yet reached the end of their visibility window.

^{— http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/sqs-metricscollected.html}

When a consumer receives a message, it has to -- at some point -- either delete the message, or send a request to increase the timeout for that message; otherwise the message becomes visible again after the timeout expires. If a consumer fails to do one of these things, the message automatically becomes visible again. The visibility timeout is how long the consumer has before one of these things must be done.

Messages should not be "in flight" without something having already received them -- but that "something" can include the console itself, as you'll note on the pop-up you see when you choose "View/Delete Messages" in the console (unless you already checked the "Don't show this again" checkbox):

Messages displayed in the console will not be available to other applications until the console stops polling for messages.

Messages displayed in the console are "in flight" while the console is observing the queue from the "View/Delete Messages" screen.

The part that does not make obvious sense is messages being in flight "for several minutes" if your default visibility timeout is only 5 seconds and nothing in your code is increasing that timeout... however... that could be explained almost perfectly by your consumers not properly disposing of the message, causing it to timeout and immediately be redelivered, giving the impression that a single instance of the message was remaining in-flight, when in fact, the message is briefly transitioning back to visible, only to be claimed almost immediately by another consumer, taking it back to in-flight again.

这篇关于为什么 SqS 消息有时会在队列中保持飞行状态的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！