Apache Camel无法及时从SQS获取消息 [英] Apache Camel not acquiring message from SQS in timely manner

问题描述

我们有一个实现，其中将消息放置在AWS SQS队列上并由Camel AWS使用.我们正在使用并发消费者=1.我们正在记录对队列的轮询，如org.apache.camel.component.aws.sqs.SqsConsumer所做的那样.

We have an implementation where messages are being placed on an AWS SQS queue and being consumed by Camel AWS. We are using concurrentConsumers = 1. We are logging the polling of the queue as done by org.apache.camel.component.aws.sqs.SqsConsumer .

该测试包括(从远程系统)向SQS发送一条消息，然后记录该消息在队列中的时间.在骆驼端，我们在SqsConsumer类上进行了跟踪日志记录，可以看到何时轮询队列以及何时使用消息.

The test consists of sending a message to SQS (from a remote system) and then logging the time the message is on the queue. On the Camel end, we have trace logging on the SqsConsumer class, and we can see when the queue is polled, and when messages are consumed.

如果我们每隔10秒将一条消息放入队列中，大多数情况下，骆驼会在1-2秒内接收到该消息.但是，很多时候它会花费更长的时间(超过10秒).

If we place a message on the queue every 10 seconds, most times Camel picks up the message in 1-2 seconds. However, there are many times when it takes significantly longer (10+ seconds).

基本上我们看到了这种行为:

Basically we see this behavior:

• (远程)在SQS上放置消息
• (骆驼)投票
• (骆驼)投票
• (骆驼)投票
• ...(对于许多民意测验，默认间隔为500毫秒)
• (骆驼)从SQS接收消息

我们已经在没有骆驼的情况下对SQS进行了端到端测试，并且吞吐量没有问题(几秒钟内有1000条消息).

We have tested SQS end to end without Camel and there is no issue with throughput (1000 messages in a few seconds).

我们针对此测试的Camel实现仅包括从SQS队列中读取并进行日志记录-没有其他功能.

Our Camel implementation for this test consists only of reading from the SQS queue and logging - there is no other functionality.

我们已经测试了更改其他许多Camel SQS参数的情况，但在行为上没有差异.

We have tested with altering many of the other Camel SQS parameters, with no difference in behavior.

但是，例如，如果我们以currentConsumConsumers = 10进行测试，则几乎立即从队列中提取每条消息，并且延迟最小.

However, if we test with concurrentConsumers = 10, for example, every message is picked up from the queue almost instantly, with minimal delay.

我的问题是，为什么单个消费者不能及时接收消息?如果它真的每500毫秒进行一次轮询，那怎么不看到"并接收消息呢?

My question is why doesn't the single consumer pick up messages in a timely manner? If it is truly polling every 500 milliseconds, how is it not "seeing" and picking up the messages?

注意-由于我们应用程序的功能，我们不愿启用并发消费所附带的多线程.

Note - we are hesitant to enable the multi-threading that comes with using concurrentConsumers because of the functionality of our application.

推荐答案

正如@ketanvijayvargiya所建议的那样，长轮询确实是答案.由于您的评论，我们对其进行了第二次查看.设置我们尝试过的值有多种方法:

As suggested by @ketanvijayvargiya, long polling indeed seems to be the answer. We took a second look at it because of your comment. There are multiple ways to set the value that we tried :

1. 通过骆驼本身通过SQS URI?receiveMessageWaitTimeSeconds =
2. 在AWS SQS队列本身上进行设置(在控制台上，配置Queue->接收消息等待时间).

我们已经测试了(2)，并且没有看到任何行为变化(在我发帖时).我们再次使用(1)进行了测试，并看到了改进.我们再次对(2)进行了重新测试，并且也看到了改进，因此我们只能假设执行第一次测试时犯了一个错误.

We had tested (2) and didn't see any behavior change (at the time of my post). We tested again using (1) and saw improvements. We retested with (2) again and also saw improvements, so we can only assume that we had made a mistake when performing the first test.

这篇关于Apache Camel无法及时从SQS获取消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！