建议对于大型单片单线程应用程序转换为多线程的架构？ [英] Advice for converting a large monolithic singlethreaded application to a multithreaded architecture?

问题描述

我公司的主要产品是大型单片C ++应用程序，用于科学数据处理和可视化。其codeBase的可以追溯到也许12或13年来，虽然我们已经把工作纳入升级和维护它（使用的STL和Boost - 当我参加了大多数容器是定制的，例如 - 全面升级为统一code和2010年VCL等），有一剩余，非常显著的问题：它是完全单线程。鉴于这是一个数据处理和可视化程序，这正成为一个障碍的越来越多了。

My company's main product is a large monolithic C++ application, used for scientific data processing and visualisation. Its codebase goes back maybe 12 or 13 years, and while we have put work into upgrading and maintaining it (use of STL and Boost - when I joined most containers were custom, for example - fully upgraded to Unicode and the 2010 VCL, etc) there's one remaining, very significant problem: it's fully singlethreaded. Given it's a data processing and visualisation program, this is becoming more and more of a handicap.

我既是一个开发和项目经理内，我们要解决这个问题，这将是在这两个领域一项困难的工作下一个版本。我正在寻找的具体的，实际的，和建筑咨询如何解决这个问题。

I'm both a developer and the project manager for the next release where we want to tackle this, and this is going to be a difficult job in both areas. I'm seeking concrete, practical, and architectural advice on how to tackle the problem.

程序的数据流可能是这样的：

The program's data flow might go something like this:

• 一个窗口，需要借助数据


• 在paint方法，它会调用一个GetData方法，往往数百次的数百个数据位在一个绘制操作


• 这将去计算或从文件或任何其他阅读要求（通常是一个相当复杂的数据流 - 认为这是流经一个复杂的图形数据，其中每个节点进行操作）

Ie的，油漆消息处理程序将被阻塞的同时处理完成，并且如果数据尚未计算并缓存，这可以是很长的时间。有时这是分钟。类似的路径发生了执行冗长的处理操作的程序的其它部分 - 该程序没有响应的整段时间，有时小时

Ie, the paint message handler will block while processing is done, and if the data hasn't already been calculated and cached, this can be a long time. Sometimes this is minutes. Similar paths occur for other parts of the program that perform lengthy processing operations - the program is unresponsive for the entire time, sometimes hours.

我正在寻找对如何处理不断变化的这个建议。切合实际的想法。也许这样的话：

I'm seeking advice on how to approach changing this. Practical ideas. Perhaps things like:

• 设计模式异步请求的数据？


• 存储对象的大集合，使得线程可以读取和写入安全？


• 处理的数据集无效，而一些试图读它？


• 有没有模式和技术为这类问题？


• 我应该会问，我都没有想到的？

我没有，因为我的Uni天做任何多线程编程，几年前，我想我的团队的其他成员处于相似的位置。我所知道的是学术，不实用，而且是隔靴搔痒有信心接近这一

I haven't done any multithreaded programming since my Uni days a few years ago, and I think the rest of my team is in a similar position. What I knew was academic, not practical, and is nowhere near enough to have confidence approaching this.

最终目标是要有一个充分响应程序，所有的计算和数据生成其他线程完成和用户界面总是有求必应。在一个单一的开发周期，我们可能无法到达那里：）

The ultimate objective is to have a fully responsive program, where all calculations and data generation is done in other threads and the UI is always responsive. We might not get there in a single development cycle :)

编辑：我想我应该添加有关应用程序一对夫妇的更多详细信息：

I thought I should add a couple more details about the app:

• 这是针对Windows 32位桌面应用程序。每个副本的许可。我们计划把它在桌面，在本地运行的应用程序


• 我们使用 Embarcadero公司（原Borland公司）C ++ Builder的2010 的发展。这会影响我们可以使用并行库，因为大多数人似乎（？）将仅用于GCC和MSVC写入。幸运的是，他们正在积极开发它和它的C ++标准的支持比以前要好得多。编译器支持这些升压组件。


• 它的架构是不是干净，因为它应该是和部件往往过于紧密耦合。这是另外一个问题：）

• It's a 32-bit desktop application for Windows. Each copy is licensed. We plan to keep it a desktop, locally-running app
• We use Embarcadero (formerly Borland) C++ Builder 2010 for development. This affects the parallel libraries we can use, since most seem (?) to be written for GCC or MSVC only. Luckily they're actively developing it and its C++ standards support is much better than it used to be. The compiler supports these Boost components.
• Its architecture is not as clean as it should be and components are often too tightly coupled. This is another problem :)

编辑＃2：感谢您​​的答复迄今

Edit #2: Thanks for the replies so far!

• 我很惊讶有这么多人推荐了多进程架构（这是目前得票最多的答案），而不是多线程。我的IM pression是这是一个非常Unix的十岁上下的程序结构，我不知道它是如何设计或作品东西。是否有良好的资源提供了有关它在Windows？难道真的那么常见的Windows？


• 在具体的方法来一些多线程的建议方面，是否有设计模式异步请求和数据的消耗，或threadaware或异步MVP系统，或者如何设计一个面向任务的系统，或者文章和书籍和后-release解构说明了工作的事情和东西不工作？我们可以开发所有这些建筑，当然我们自己，但它是很好的从别人以前做过的工作，知道要避免什么错误和缺陷。


• 未在任何答案触及一个方面是项目管理这个。我的IM pression被估计这将花费的时间和做的不确定性，因为这东西的时候可能很难保证项目的良好控制。这是原因之一配方或实际编码意见后，我，我猜，引导和限制编码方向尽可能。

我还没有标记为这个问题的答案 - 这是因为答案的质量，这是伟大的不是（三江源），而只是说，由于这个，我希望更多的答案的范围或讨论。三江源那些谁已经回答！

I haven't yet marked an answer for this question - this is not because of the quality of the answers, which is great (and thankyou) but simply that because of the scope of this I'm hoping for more answers or discussion. Thankyou to those who have already replied!

推荐答案

所以，有一个在你的算法描述的提示，如何进行：

So, there's a hint in your description of the algorithm as to how to proceed:

通常一个相当复杂的数据流 - 认为这是流经一个复杂的图形数据，每个节点都执行操作

often quite a complex data flow - think of this as data flowing through a complex graph, each node of which performs operations

我会考虑做数据流图是从字面上，做工作的结构。图中的链接可以是线程安全队列，每个节点的算法可以住pretty多不变，除了包在从上一个队列和存款的结果拾取工作项的线程。你可以再前进一步，并使用插座和流程，而不是队列和线程;这会让你正穿过多台机器$ P $垫如果在这样一个性能优势。

I'd look into making that data-flow graph be literally the structure that does the work. The links in the graph can be thread-safe queues, the algorithms at each node can stay pretty much unchanged, except wrapped in a thread that picks up work items from a queue and deposits results on one. You could go a step further and use sockets and processes rather than queues and threads; this will let you spread across multiple machines if there is a performance benefit in doing this.

然后您的油漆和其他GUI方法都需要一分为二：一半排队的工作，另一半绘制或者他们出来管道的使用效果。

Then your paint and other GUI methods need split in two: one half to queue the work, and the other half to draw or use the results as they come out of the pipeline.

这可能不是实际的，如果该应用presumes数据是全球性的。但是，如果人们都包含在类，你的描述表明，它可能是，那么这可能是得到它并行化的最简单方法。

This may not be practical if the app presumes that data is global. But if it is well contained in classes, as your description suggests it may be, then this could be the simplest way to get it parallelised.

这篇关于建议对于大型单片单线程应用程序转换为多线程的架构？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！