为什么 multiprocessing.Process 在 windows 和 linux 上对于全局对象和函数参数的行为不同 [英] Why multiprocessing.Process behave differently on windows and linux for global object and function arguments
问题描述
下面的代码在windows和linux(都是python2.7)上运行时输出不同
The following code has different output when running on windows and linux (both with python2.7)
'''import_mock.py'''
to_mock = None
'''test.py'''
import import_mock
from multiprocessing import Process
class A(object):
def __init__(self):
self.a = 1
self.b = 2
self.c = 3
def __getstate__(self):
print '__getstate__'
return { 'a': self.a, 'b': self.b,
'c':0 }
def func():
import_mock.to_mock = 1
a = A()
return a
def func1(a):
print a.a, a.b, a.c
print import_mock.to_mock
if __name__ == '__main__':
a = func()
p = Process(target=func1, args=(a,))
p.start()
p.join()
在 windows 上，输出为:
On windows, the output is:
__getstate__
1 2 0
None
这是我的预期
在linux上是:
1 2 3
1
不克隆全局对象和传递的参数.
Which not clone the global object and the passed args.
我的问题是为什么他们的行为不同?以及如何使 linux 代码的行为与 windows one 相同?
My question is why they behave differently? And how to make the linux code behave the same as windows one?
推荐答案
补充@Blckknght 的回答:在 Windows 上，每个进程从头开始"导入原始模块，而在 Unix-y 系统上只有主进程运行整个模块，而所有其他进程看到在 fork() 用于创建新进程时存在的任何内容(不，你没有调用 fork() 自己 - multiprocessing 内部在创建新进程时调用它).
Adding to @Blckknght's answer: on Windows, each process imports the original module "from scratch", while on Unix-y systems only the main process runs the whole module, while all other processes see whatever exists at the time fork() is used to create the new processes (no, you're not calling fork() yourself - multiprocessing internals call it whenever it creates a new process).
详细来说，对于您的 import_mock:
In detail, for your import_mock:
在所有平台上，主进程调用
func()，将import_mock.to_mock设置为1.
在 Unix-y 平台上，这是所有新进程所看到的:fork() 发生在之后，因此 1 是所有新进程继承的状态.
On Unix-y platforms, that's what all new processes see: the fork() occurs after that, so 1 is the state all new processes inherit.
在 Windows 上，所有新进程从头开始"运行整个模块.所以他们每个人都导入了自己的全新版本的 import_mock.只有主进程调用 func()，所以只有主进程看到 to_mock 变为 1.所有其他进程看到新的 None 状态.
On Windows, all new processes run the entire module "from scratch". So they each import their own, brand new version of import_mock. Only the main process calls func(), so only the main process sees to_mock change to 1. All other processes see the fresh None state.
这一切都在意料之中，实际上很容易理解第二次;-)
That's all expected, and actually easy to understand the second time ;-)
传递 a 的情况更为微妙，因为它更多地取决于 multiprocessing 实现细节.实现本可以从一开始就选择在所有平台上腌制参数，但它没有，现在在一些平台上不破坏东西的情况下改变已经太晚了.
What's going on with passing a is subtler, because it depends more on multiprocessing implementation details. The implementation could have chosen to pickle arguments on all platforms from the start, but it didn't, and now it's too late to change without breaking stuff on some platforms.
由于写时复制 fork() 语义，在 Unix 上腌制 Process() 参数不是必要 -y 系统，因此实施从未如此.但是，如果没有 fork()，则必须在 Windows 上腌制它们 - 实现也是如此.
Because of copy-on-write fork() semantics, it wasn't necessary to pickle Process() arguments on Unix-y systems, and so the implementation never did. However, without fork() it is necessary to pickle them on Windows - and so the implementation does.
在允许您在所有平台上强制Windows 实现"(spawn)的 Python 3.4 之前，没有机械方法可以避免可能的跨平台意外.
Before Python 3.4, which allows you to force "the Windows implementation" (spawn) on all platforms, there's no mechanical way to avoid possible cross-platform surprises.
但在实践中，我很少对此感到困扰.知道，例如，多处理可能严重依赖于酸洗，我完全不知道在任何地方玩泡菜的把戏.您在传递 A() 实例时遇到问题"的唯一原因是您在玩泡菜技巧(通过覆盖默认的 __getstate__()).
But in practice, I've rarely been bothered by this. Knowing that, for example, multiprocessing can depend heavily on pickling, I stay completely clear of getting anywhere near playing tricks with pickles. The only reason you had "a problem" passing an A() instance is that you are playing pickle tricks (via overriding the default __getstate__()).
这篇关于为什么 multiprocessing.Process 在 windows 和 linux 上对于全局对象和函数参数的行为不同的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
