多处理池的意外行为 [英] Unexpected behavior with multiprocessing Pool

问题描述

在下面的代码中，我期望输出为 2，因为我在将函数分配给池进行多处理之前更改了 config 的值，但我得到的是 5.我确信有一个很好的理由为它，但不知道如何解释它.

In the code below, I was expecting the output to be 2 as I'm changing the value of config before assigning function to pool for multiprocessing, but instead I'm getting 5. I'm sure there is a good reason for it, but not sure how to explain it.

from multiprocessing import Pool 
config = 5

class Test:

  def __init__(self):
    print("This is init")

  @classmethod
  def testPrint(cls, data):
    print(config)
    print("This is testPrint")
    return config

if __name__ == "__main__" :
  pool = Pool()
  config = 2
  output = pool.map(Test.testPrint, range(10))
  print(output)

输出

5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
5
This is testPrint
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5]

推荐答案

创建池时会创建新进程.之后，对父内存空间所做的更改，除了在像 .map 这样的池函数中传递的内容之外，子进程都看不到.像 linux 这样的分叉系统会为父内存空间创建写时复制视图 - 并且该写入会为父进程生成一个唯一的内存块，子进程看不到.生成系统重新导入模块(设置全局变量)，然后尝试为子进程pickle/unpickle 状态.在这两种情况下，这都在 Pool 类初始化返回到您的程序之前完成.

The new processes are created when you create the pool. After that, changes made to the parent memory space, except for stuff that is passed in a pool function like .map, are not seen by the subprocess. Forking systems like linux create copy-on-write views to the parent memory space - and that write results in a unique memory block for the parent, not seen by the subprocess. Spawning systems reimport modules (setting global variables) and then try to pickle/unpickle state for the subprocesses. In both cases, this is completed before the Pool class initialization returns to your program.

这篇关于多处理池的意外行为的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！