如何在不使用Web仪表板的情况下剖析Apache Storm拓扑? [英] How can I profile Apache Storm topologies without using the web dashboard?

问题描述

标题几乎说明了一切，我有一些Storm拓扑，我想衡量它们的延迟，即从Kafka发出消息到最终执行相关操作之间的时间量螺栓.如果我可以深入研究结果以查看每个螺栓的延迟，则可以加分.

The title pretty much says it all, I have some Storm topologies and I'd like to measure their latencies, that is, the amount of time between a message coming in from Kafka and the last bit of related execution in the final bolt. Bonus points if I can drill into the results to see the latency across each bolt.

能否仅通过调整Storm配置来完成此操作?

Can this be done by simply tweaking the Storm configuration?

如果不是，则为 http://storm. incubator.apache.org/apidocs/backtype/storm/hooks/info/SpoutAckInfo.html backtype.storm.hooks.info.SpoutAckInfo.completeLatency是我要找的东西吗?

If not, is http://storm.incubator.apache.org/apidocs/backtype/storm/hooks/info/SpoutAckInfo.html backtype.storm.hooks.info.SpoutAckInfo.completeLatencyMs the thing I'm looking for?

推荐答案

对于常规性能分析，我们遇到了麻烦，我可以让管理员动态设置并打开比工作端口(工作端口的端口)高1000的JMX端口.当然，这限制了我们可以分配的工作人员数量，但是我们现在离这个问题还很遥远).然后，使用supervisor.childopts值分配工作人员的JMX端口并连接到Visual VM.

For general profiling, we have a fork of storm where I enable the supervisor to dynamically set and open the JMX port equal to 1000 higher than the worker port (of course, this limits the number of workers we could assign, but we're nowhere near that problem now). Then with the supervisor.childopts value we assign the worker's JMX port and connect with Visual VM.

针对您的指标问题，您可以使用要点.请注意，我们尚未将codahale指标提高到3.x.

Specific to your metrics question, you can use the BaseTaskHook to push the storm metrics to something with an MBean (we use codahale metrics) and again, browse the values over JMX. I posted an example gist. Note we haven't uplifted to 3.x of codahale metrics.

相信使用Java 7工具，您可以在运行中的JVM上打开JMX ，因此，如果您使用的是Java 7+，这也将为您提供帮助.

With Java 7 tools, I believe you're able to turn on JMX on a running JVM, so if you're on Java 7+, that would also help you.

这篇关于如何在不使用Web仪表板的情况下剖析Apache Storm拓扑?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！