Python多处理的Pool进程限制 [英] Python multiprocessing's Pool process limit
问题描述
在使用多处理模块中的Pool对象时，进程数是否受CPU内核数的限制?例如.如果我有4个核心，即使我创建具有8个进程的Pool，一次也只能运行4个?
In using the Pool object from the multiprocessing module, is the number of processes limited by the number of CPU cores? E.g. if I have 4 cores, even if I create a Pool with 8 processes, only 4 will be running at one time?
推荐答案
您可以根据需要请求多个进程.可能存在的任何限制将由您的操作系统而不是multiprocessing施加.例如，
You can ask for as many processes as you like. Any limit that may exist will be imposed by your operating system, not by multiprocessing. For example,
p = multiprocessing.Pool(1000000)
在任何机器上都可能遭受丑陋的死亡.我在输入内容时在盒子上尝试，操作系统正在磨碎我的磁盘，以疯狂地交换掉RAM-在它创建了大约3000个进程之后最终将其杀死；-)
is likely to suffer an ugly death on any machine. I'm trying it on my box as I type this, and the OS is grinding my disk to dust swapping out RAM madly - finally killed it after it had created about 3000 processes ;-)
关于一次"将运行多少，Python对此没有发言权.这取决于:
As to how many will run "at one time", Python has no say in that. It depends on:
- 能够运行的硬件有多少?并且，
- 您的操作系统如何决定为当前正在运行的计算机上的进程 all 提供硬件资源.
- How many your hardware is capable of running simultaneously; and,
- How your operating system decides to give hardware resources to all the processes on your machine currently running.
对于CPU密集型任务，创建所需的Pool进程不会使 sense 超出您具有运行它们的核心的能力.如果您也尝试将计算机用于其他用途，则创建的进程应少于核心.
For CPU-bound tasks, it doesn't make sense to create more Pool processes than you have cores to run them on. If you're trying to use your machine for other things too, then you should create fewer processes than cores.
对于受I/O约束的任务， 创建比内核更多的Pool进程可能是有意义的，因为这些进程可能会花费大部分时间来阻塞(等待I /O完成).
For I/O-bound tasks, it may make sense to create a quite a few more Pool processes than cores, since the processes will probably spend most their time blocked (waiting for I/O to complete).
这篇关于Python多处理的Pool进程限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
