使用 multiprocessing.Process 与 subprocess.Popen 的速度比较 [英] Speed comparison using multiprocessing.Process versus subprocess.Popen

问题描述

我正在使用 Python3 执行 PYQT 代码；同时，我需要调用Python2.7代码，对于Python3无法执行的操作.

I am using Python3 to execute PYQT code; and at the same time, I need to call Python2.7 code, for operations that I cannot perform via Python3.

我确实通过 Popen 实现了 2.7 代码执行；尽管从 Popen 调用时，运行 2.7 代码需要相当长的时间.如果我直接从 Python2.7 运行相同的操作，执行速度会快得多.

I did implement the 2.7 code execution via Popen; although it takes a considerable amount of time to run the 2.7 code, when called from Popen. The same operation is performed much faster if I run it directly from Python2.7.

是否可以使用 multiprocessing 而不是 subprocess.Popen 来达到同样的目的，以加快 2.7 代码的执行速度?如果合适的话；从 multiprocessing.Process 调用 Python2.7 代码的正确方法是什么?还是使用多进程是一种浪费，因为我只执行一个操作?

Would be possible to use multiprocessing instead of subprocess.Popen for the same purpose, to speed up the execution of the 2.7 code? And if that is appropriate; what would be the correct way to call Python2.7 code from a multiprocessing.Process? Or is it a waste to use multiprocess, since I am executing only one operation?

推荐答案

multiprocessing 与 subprocess 类似，仅在不能 fork 进程，这样理论上你就可以破解multiprocessingcode> 使用不同的解释器.这会比它的价值更麻烦，因为那时你不会在产生 subprocess 和使用 multiprocessing.Process 之间获得任何性能提升(实际上，由于 multiprocessing.Process 增加了通信开销，它可能会以较慢的速度结束.

multiprocessing is similar to subprocess only on non-POSIX systems that cannot fork processes so you could, theoretically, hack away multiprocessing to use a different interpreter. It would be more trouble than its worth, tho, because at that point you wouldn't get any performance boost between spawning a subprocess and using a multiprocessing.Process (in fact, it would probably end slower due to the communication overhead added to multiprocessing.Process).

因此，如果我们只讨论必须在不同解释器中执行的单个任务，这将与您获得的一样快.如果要在不同的解释器中执行多个任务，您仍然可以从 multiprocessing.Process 中受益，方法是生成一个子进程来运行不同的解释器，然后在其中使用 multiprocessing在您的核心上分配多个任务.

So, if we're talking only about a single task that has to execute in a different interpreter, this is as fast as you're gonna get. If there are multiple tasks to be executed in a different interpreter you may still benefit from multiprocessing.Process by spawning a single subprocess to run the different interpreter and then using multiprocessing within it to distribute multiple tasks over your cores.

这篇关于使用 multiprocessing.Process 与 subprocess.Popen 的速度比较的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！