Kinesis 流待处理消息计数 [英] Kinesis stream pending message count

问题描述

我正在尝试将 AWS Kinesis 流用于我们的数据流之一.我想出于操作目的监视我的流中的待处理消息(根据积压向下游扩展)，但无法找到任何在我的流中提供(大约)待处理消息的 API.

I am trying to use AWS Kinesis stream for one of our data streams. I would like to monitor pending messages on my stream for ops purposes(scale downstream according to backlog), but unable to find any API that gives (approx) pending messages in my stream.

这看起来很奇怪，因为消息在 7 天后过期，如果生产者和消费者被隔离并且无法通信，你怎么知道消息即将过期.你如何处理这个问题?

This looks strange as messages get expired after 7 days and if the producers and consumers are isolated and can't communicate, how do you know messages are expiring. How do you handle this problem?

谢谢！

推荐答案

Kinesis 中没有待处理"消息这样的概念.所有传入的数据都将放在一个分片上.

There is no such concept as "pending" message in Kinesis. All the incoming data will be placed on a shard.

您的消费者应用程序应始终处于运行状态，以跟踪流中的变化.应用程序(在 KCL 的帮助下)将在后台继续轮询Shard Iterator"，因此您会在新数据到来时收到通知.

Your consumer application should be in running state all the time, to keep track of changes in your stream. The application (with the help of KCL) will continue to poll "Shard Iterator" in the background, thus you will be notified about the new data when it comes.

大致;您可以将 Kinesis 视为一个 FIFO 队列，如果您不弹出它们，消息将在短时间内消失.

Roughly; you can see Kinesis as a FIFO queue and the messages will disappear in a short time if you don't pop them.

如果您的应用程序将在一个小时内处理几条消息，您应该考虑更改您的架构.Kinesis 可能不是适合您的工具.

If your application will process a few messages in an hour, you should think about changing your architecture. Kinesis is probably not the correct tool for you.

这篇关于Kinesis 流待处理消息计数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！