如何可视化许多并发多阶段过程的行为？ [英] How to visualize the behavior of many concurrent multi-stage processes?
问题描述
假设我有一吨（连续流）的请求要处理，每个请求有几个阶段。例如：连接到数据源，从数据源读取数据，验证数据，处理数据，连接到数据接收器，写入结果到数据接收器。
哪些可视化方法甚至工具适合用于可视化此类系统的行为？
我想要能够看到阶段需要很长时间，以及不同请求的阶段如何相对于彼此对齐（例如，查看数据源在由太多请求一次访问时响应更长）。
如果只有几十个请求，我可以用几十个单独的彩色时间表，但对于几千个不适合。我想我可以用N个颜色的时间线，其中N是并发因素，但1）也许有一些更好的东西，2）可能有这样的工具吗？
PS无耻的插件：一旦我找出最好的可视化方式，我将它添加到我的漂亮的工具，名为。要保持注意什么是记录什么时候和模式我使用syslog查看器。有很多你可以找到一个适合你。 http ：//www.google.com/search？aq = 0& oq = syslog + vi& sourceid = chrome& ie = UTF-8& q = syslog + viewer
希望这有帮助。
Suppose I've got a ton (a continuous stream) of requests to process, and each request has several stages. For example: "connecting to data source", "reading data from data source", "validating data", "processing data", "connecting to data sink", "writing result to data sink".
Which visualization methods or even tools fit well to visualize the behavior of such a system?
I'd like to be able to see which stages are taking a long time, and how the stages of different requests are aligned with respect to each other (for example, to see that the data source responds longer when accessed by too many requests at once).
If there were just a few dozen requests, I'd be OK with a few dozen individual colored timelines, but for a few thousand that doesn't fit well. I think I can get away with N colored timelines, where N is the "concurrency factor", but 1) perhaps there's something better, 2) perhaps there exist tools for this?
P.S. Shameless plug: Once I figure out the best way of visualization, I'll add it to my nifty tool called timeplot ;)
P.P.S. Another shameless plug: I decided to write a separate tool: splot. Here's what it can do, based on a trivially simple log and an awk one-liner:
It's showing 160 cores of a cluster performing tasks fed to them by RabbitMQ. Blue is "fetching data", orange is "computing", white is "doing nothing". Several problems are immediately obvious from this diagram, which would be very hard to find by just looking at the logs.
I have a multi process software that runs on a machine with 15 cores. Here is what I do.
Log all messages to syslog. Finally plot (selected) last 20 minutes log data on http://www.simile-widgets.org/timeline. To keep an eye on what is getting logged when and the patterns I use syslog viewer. There are plenty you can find the one that suits you. http://www.google.com/search?aq=0&oq=syslog+vi&sourceid=chrome&ie=UTF-8&q=syslog+viewer
Hope this helps.
这篇关于如何可视化许多并发多阶段过程的行为？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
