如何使用Python中的boto库获取Amazon SQS队列中的所有消息? [英] How to get all messages in Amazon SQS queue using boto library in Python?

问题描述

我正在开发一个应用程序，该应用程序的工作流是通过使用Boto在SQS中传递消息来管理的.

I'm working on an application whose workflow is managed by passing messages in SQS, using boto.

我的SQS队列正在逐渐增长，我无法检查它应该包含多少个元素.

My SQS queue is growing gradually, and I have no way to check how many elements it is supposed to contain.

现在，我有一个守护程序，可以定期轮询队列，并检查是否有固定大小的元素集.例如，考虑以下队列":

Now I have a daemon that periodically polls the queue, and checks if i have a fixed-size set of elements. For example, consider the following "queue":

q = ["msg1_comp1", "msg2_comp1", "msg1_comp2", "msg3_comp1", "msg2_comp2"]

现在我想检查在某个时间点队列中是否同时存在"msg1_comp1"，"msg2_comp1"和"msg3_comp1"，但是我不知道队列的大小.

Now I want to check if I have "msg1_comp1", "msg2_comp1" and "msg3_comp1" in the queue together at some point in time, but I don't know the size of the queue.

浏览API后，您似乎只能在队列中获得1个元素或固定数量的元素，但并非全部:

After looking through the API, it seems you can either get only 1 element, or a fixed number of elements in the queue, but not all:

>>> rs = q.get_messages()
>>> len(rs)
1
>>> rs = q.get_messages(10)
>>> len(rs)
10

答案中建议的一个建议是循环获取10条消息，直到我什么都没收到为止，但是SQS中的消息具有可见性超时，这意味着如果我从队列中轮询元素，它们将不会真正移除后，它们只会在短时间内不可见.

A suggestion proposed in the answers would be to get for example 10 messages in a loop until I get nothing back, but messages in SQS have a visibility timeout, meaning that if I poll elements from the queue, they won't be really removed, they will only be invisible for a short period of time.

是否有一种简单的方法来获取队列中的所有消息，而又不知道有多少?

Is there a simple way to get all messages in the queue, without knowing how many there are?

推荐答案

在while循环内将您的呼叫置于q.get_messages(n):

Put your call to q.get_messages(n) inside while loop:

all_messages=[]
rs=q.get_messages(10)
while len(rs)>0:
    all_messages.extend(rs)
    rs=q.get_messages(10)

此外，转储将不支持超过10条消息之一:

def dump(self, file_name, page_size=10, vtimeout=10, sep='\n'):
    """Utility function to dump the messages in a queue to a file
    NOTE: Page size must be < 10 else SQS errors"""

这篇关于如何使用Python中的boto库获取Amazon SQS队列中的所有消息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！