为什么SqS消息有时仍在队列中进行中 [英] Why do SqS messages sometimes remain in-flight on queue

问题描述

我以一种非常简单的方式使用Amazon SQS队列.通常，消息被写入并立即可见和读取.有时，会写入一条消息，并在队列中保持飞行中(不可见)几分钟.我可以从控制台看到它.接收消息等待时间为0，默认可见性为5秒.它将保持这种状态几分钟，或者直到写入新消息以某种方式释放它为止.几秒钟的延迟是可以的，但超过60秒的延迟是不可以的.

有8个读取器线程始终长时间轮询，因此并不是某些人没有尝试读取它.

编辑:为清楚起见，所有使用者阅读都没有返回任何消息，并且无论控制台是否打开，它都会发生.在这种情况下，只涉及一条消息，它只是位于对消费者不可见的队列中.

有没有其他人看到过这种行为以及我可以做些什么来改善它?

这是我正在使用的Java的sdk:

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk</artifactId>
  <version>1.5.2</version>
</dependency>     

以下是执行读取操作的代码(max = 10，maxwait = 0启动配置):

void read(MessageConsumer consumer) {

  List<Message> messages = read(max, maxWait);

  for (Message message : messages) {
    if (tryConsume(consumer, message)) {
      delete(message.getReceiptHandle());
    }
  }
}

private List<Message> read(int max, int maxWait) {

  AmazonSQS sqs = getClient();
  ReceiveMessageRequest rq = new ReceiveMessageRequest(queueUrl);
  rq.setMaxNumberOfMessages(max);
  rq.setWaitTimeSeconds(maxWait);
  List<Message> messages = sqs.receiveMessage(rq).getMessages();

  if (messages.size() > 0) {
    LOG.info("read {} messages from SQS queue",messages.size());
  }

  return messages;
}

"read .."的日志行永远不会出现，这是导致我进入控制台并查看消息是否存在的原因.

解决方案

听起来您误解了所看到的内容.

飞行中"消息不是待处理的传递，它们是已经传递但消费者未采取进一步行动的消息.

如果消息已发送到客户端但尚未删除或尚未到达可见性窗口的末尾，则视为正在运行.

^{- http://docs.aws. amazon.com/AmazonCloudWatch/latest/monitoring/sqs-metricscollected.html}

当消费者收到一条消息时，它必须(在某个时候)删除该消息或将请求发送到

在控制台停止轮询消息之前，控制台中显示的消息将对其他应用程序不可用.

控制台中显示的消息正在运行"，当控制台正在从查看/删除消息"中观察队列时，屏幕.

没有明显意义的部分是消息在飞行中持续几分钟".如果您的默认可见性超时仅为5秒，而您的代码中没有任何东西可以增加该超时...但是...这可以由您的消费者没有正确处理该消息，导致该消息超时并立即重新传递而做出的几乎完美的解释，给人的印象是该消息的单个实例仍在运行中，而实际上，该消息只是短暂地过渡回可见状态，几乎被另一位消费者几乎立即声明，然后又重新返回到运行中.

I'm using Amazon SQS queues in a very simple way. Usually, messages are written and immediately visible and read. Occasionally, a message is written, and remains In-Flight(Not Visible) on the queue for several minutes. I can see it from the console. Receive-message-wait time is 0, and Default Visibility is 5 seconds. It will remain that way for several minutes, or until a new message gets written that somehow releases it. A few seconds delay is ok, but more than 60 seconds is not ok.

There a 8 reader threads that are long polling always, so its not that something is not trying to read it, they are.

Edit : To be clear, none of the consumer reads are returning any messages at all and it happens regardless of whether or not the console is open. In this scenario, only one message is involved, and it is just sitting in the queue invisible to the consumers.

Has anyone else seen this behavior and what I can do to improve it?

Here is the sdk for java I am using:

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk</artifactId>
  <version>1.5.2</version>
</dependency>     

Here is the code that does the reading (max=10,maxwait=0 startup config):

void read(MessageConsumer consumer) {

  List<Message> messages = read(max, maxWait);

  for (Message message : messages) {
    if (tryConsume(consumer, message)) {
      delete(message.getReceiptHandle());
    }
  }
}

private List<Message> read(int max, int maxWait) {

  AmazonSQS sqs = getClient();
  ReceiveMessageRequest rq = new ReceiveMessageRequest(queueUrl);
  rq.setMaxNumberOfMessages(max);
  rq.setWaitTimeSeconds(maxWait);
  List<Message> messages = sqs.receiveMessage(rq).getMessages();

  if (messages.size() > 0) {
    LOG.info("read {} messages from SQS queue",messages.size());
  }

  return messages;
}

The log line for "read .." never appears when this is happening, and its what causes me to go in with the console and see if the message is there or not, and it is.

解决方案

It sounds like you are misinterpreting what you are seeing.

Messages "in flight" are not pending delivery, they're messages that have already been delivered but not further acted on by the consumer.

Messages are considered in flight if they have been sent to a client but have not yet been deleted or have not yet reached the end of their visibility window.

^— http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/sqs-metricscollected.html

When a consumer receives a message, it has to -- at some point -- either delete the message, or send a request to increase the timeout for that message; otherwise the message becomes visible again after the timeout expires. If a consumer fails to do one of these things, the message automatically becomes visible again. The visibility timeout is how long the consumer has before one of these things must be done.

Messages should not be "in flight" without something having already received them -- but that "something" can include the console itself, as you'll note on the pop-up you see when you choose "View/Delete Messages" in the console (unless you already checked the "Don't show this again" checkbox):

Messages displayed in the console will not be available to other applications until the console stops polling for messages.

Messages displayed in the console are "in flight" while the console is observing the queue from the "View/Delete Messages" screen.

The part that does not make obvious sense is messages being in flight "for several minutes" if your default visibility timeout is only 5 seconds and nothing in your code is increasing that timeout... however... that could be explained almost perfectly by your consumers not properly disposing of the message, causing it to timeout and immediately be redelivered, giving the impression that a single instance of the message was remaining in-flight, when in fact, the message is briefly transitioning back to visible, only to be claimed almost immediately by another consumer, taking it back to in-flight again.

这篇关于为什么SqS消息有时仍在队列中进行中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！