Kinesis流待处理消息计数 [英] Kinesis stream pending message count

问题描述

我正在尝试将AWS Kinesis流用于我们的数据流之一.我想出于操作目的监视流中的未决消息(根据积压向下游扩展)，但是找不到在我的流中给出(大约)未决消息的任何API.

I am trying to use AWS Kinesis stream for one of our data streams. I would like to monitor pending messages on my stream for ops purposes(scale downstream according to backlog), but unable to find any API that gives (approx) pending messages in my stream.

这看起来很奇怪，因为消息在7天后过期，并且如果生产者和消费者处于隔离状态并且无法通信，您如何知道消息即将过期.您如何处理这个问题?

This looks strange as messages get expired after 7 days and if the producers and consumers are isolated and can't communicate, how do you know messages are expiring. How do you handle this problem?

谢谢！

推荐答案

Kinesis中没有待处理"消息之类的概念.所有传入的数据都将放置在分片上.

There is no such concept as "pending" message in Kinesis. All the incoming data will be placed on a shard.

您的使用者应用程序应始终处于运行状态，以跟踪流中的更改.该应用程序(在KCL的帮助下)将继续在后台轮询碎片迭代器"，因此，当新数据出现时，您将收到通知.

Your consumer application should be in running state all the time, to keep track of changes in your stream. The application (with the help of KCL) will continue to poll "Shard Iterator" in the background, thus you will be notified about the new data when it comes.

大致上；您可以将Kinesis视为FIFO队列，如果不弹出消息，消息将在短时间内消失.

Roughly; you can see Kinesis as a FIFO queue and the messages will disappear in a short time if you don't pop them.

如果您的应用程序在一小时内将处理几则消息，则应考虑更改体系结构. Kinesis可能不是您的正确工具.

If your application will process a few messages in an hour, you should think about changing your architecture. Kinesis is probably not the correct tool for you.

这篇关于Kinesis流待处理消息计数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！