为什么多个流程会变慢？ [英] Why do multiple processes slow down?

问题描述

不确定这是这个问题的最佳标题，但是这里有。

Not sure this is the best title for this question but here goes.

通过python / Qt我启动了多个可执行文件的进程。每个进程都以块的形式将大文件（~20GB）写入磁盘。我发现第一个开始的过程总是最后一个完成并持续很长时间，比其他过程要长得多（尽管要写入相同数量的数据）。

Through python/Qt I started multiple processes of an executable. Each process is writing a large file (~20GB) to disk in chunks. I am finding that the first process to start is always the last to finish and continues on much, much longer than the other processes (despite having the same amount of data to write).

性能监视器显示该进程仍在使用预期的RAM量（~1GB），但进程的磁盘活动已经减慢到涓流。

Performance monitors show that the process is still using the expected amount of RAM (~1GB), but the disk activity from the process has slowed to a trickle.

为什么会发生这种情况？就好像第一个进程以某种方式开始被其他进程阻止'磁盘访问'，然后在其他进程完成后无法恢复...

Why would this happen? It is as though the first process started somehow gets its' disk access 'blocked' by the other processes and then doesnt recover after the other processes have finished...

将操作系统（Windows）会造成这种情况吗？我该怎么做才能缓解这种情况？

Would the OS (windows) be causing this? What can I do to alleviate this?

推荐答案

并行（任何类型）只会导致加速，如果你真的有更好的资源来解决问题。

在考虑优化程序之前，你应该仔细分析导致它运行（主观上）慢的原因 - 瓶颈。

Before thinking of optimizing your program, you should carefully analyze what's causing it to run (subjectively) slow - the bottleneck.

虽然我对你的程序有什么样的瓶颈一无所知，但它写得很大磁盘数据量是一个很好的提示，它可能是 I / O绑定。

While I know nothing about what sort bottleneck your program has, the fact that it writes a large quantity of data to disk is a good hint that it may be I/O bound.

当一个程序被I / O绑定时，传统的单机并行化技术（线程，多个进程）比无用的更糟糕 - 它们法案尤其是如果你正在处理旋转磁盘，这会伤害到性能。发生这种情况是因为一旦您有多个进程在不同位置访问磁盘，硬盘驱动器头必须在这些之间寻找。

When a program is I/O bound, the conventional single-machine parallelization techniques (threading, multiple processes) are worse than useless - they actually hurt performance, especially if you're dealing with a spinning disk. This happens because once you have more than one process accessing the disk at different places, the hard drive head has to seek between those.

I / O调度程序可以对多个进程访问I / O后性能变慢的影响产生很大影响，以及如何分配进程的磁盘访问。您可以考虑切换您的操作系统，但仅限于首先需要所需的多个进程

The I/O scheduler of your OS can have a great impact on how slower performance becomes once you have multiple processes accessing I/O, and how processes are alloted disk accesses. You may consider switching your OS, but only if those multiple processes are needed in the first place.

话虽如此，你可以 做些什么才能获得更好的（I / O）性能？

With that being said, what can you do to get better (I/O) performance?

• 获得更好的磁盘（或SSD）


• 获取更多磁盘（每个进程一个）


• 获取更多机器

这篇关于为什么多个流程会变慢？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！