如何监控 Bolt 的待处理队列的大小? [英] How to monitor the size of Bolt's pending queues?

问题描述

我的拓扑有一两个瓶颈.Storm UI 中的容量指标对于识别这些非常有用，但我对 Bolt 队列的大小更感兴趣.

My topology has a bottleneck or two. The capacity metric in the Storm UI is useful for identifying these, but I'd be much more interested in the size of Bolt's queues.

我的理解是每个 bolt 有两个队列，一个用于待执行的元组，另一个用于待发出的元组.是否可以监控这些队列的大小?

My understanding is that each bolt has two queues, one for tuples pending to be executed, and another for tuple pending to be emitted. Is it possible to monitor the size of these queues?

我在网上找到了一些关于向 Bolts 添加 ITaskHook 实现的内容，但目前还不清楚如何使用它来监控队列大小.可以使用 ITaskHook 中的方法来监控吗?

I found some stuff online about adding an ITaskHook implementation to Bolts, but it's not remotely clear how I can use this to monitor queue size. Can the methods in ITaskHook be used for to monitor this?

推荐答案

您应该能够使用 metrics 机制.一个简单的方法是将 conf.registerMetricsConsumer(LoggingMetricsConsumer.class) 添加到拓扑配置中.

You should be able to see the length of the queues for the components of your topology using the metrics mechanism. An easy way of doing is to add conf.registerMetricsConsumer(LoggingMetricsConsumer.class) to the config of your topology.

这是我从我的一个组件中获得的示例

Here is an example of what I get for one of my components

4:fetch __sendqueue {write_pos=12122, read_pos=12122, capacity=1024,population=0}4:fetch __receive {write_pos=8588, read_pos=8587, capacity=1024,population=1}

这篇关于如何监控 Bolt 的待处理队列的大小?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！