设置env var在Python中的多进程处理 [英] set env var in Python multiprocessing.Process

问题描述

在子进程 Python 2模块中， Popen 可以被赋予一个 env 。

In the subprocess Python 2 module, Popen can be given an env.

似乎等同于使用多处理模块中的rel =nofollow>流程模块是在 args 或 kwargs 中传递 env dictionnary，然后在目标中使用 os.environ ['FOO'] = value 。

Seems that the equivalent way to do it with Process in multiprocessing module is to pass the env dictionnary in args or kwargs, and then use os.environ['FOO'] = value in the target.

是正确的方式吗？

是否安全？我的意思是，没有风险可以修改父进程或其他子进程中的环境？

Is it safe? I mean, no risk that the environment in the parent process or other child processes can be modified?

这是一个例子（可以使用）。

Here is an example (that works).

import multiprocessing
import time
import os

def target(someid):
    os.environ['FOO'] = "foo%i" % someid
    for i in range(10):
        print "Job %i: " % someid, os.environ['FOO']
        time.sleep(1)

if __name__ == '__main__':

    processes = []

    os.environ['FOO'] = 'foo'

    for someid in range(3):
        p = multiprocessing.Process(target=target, args=(someid,))
        p.start()
        processes.append(p)

    for i in range(10):
        print "Parent: ", os.environ['FOO']
        time.sleep(1)

    for p in processes:
        p.join()

推荐答案

是的，这是正确的方法。虽然孩子将从父母继承其初始环境，但是在小孩中进行的 os.environ 的更改将不会影响父级，反之亦然：

Yes, that's the right way to do it. While the child will inherit its initial environment from the parent, subsequent changes to os.environ made in the child will not affect the parent, and vice-versa:

import os
import multiprocessing


def myfunc(q):
    print "child: " + os.environ['FOO']
    os.environ['FOO'] = "child_set"
    print "child new: " + os.environ['FOO']
    q.put(None)
    q.get()
    print "child new2: " + os.environ['FOO']


if __name__ == "__main__":
    os.environ['FOO'] = 'parent_set'
    q = multiprocessing.Queue()
    proc = multiprocessing.Process(target=myfunc, args=(q,))
    proc.start()
    q.get()
    print "parent: " + os.environ['FOO']
    os.environ['FOO'] = "parent_set_again"
    q.put(None)

输出：

child start: parent_set
child after changing: child_set
parent after child changing: parent_set
child after parent changing: child_set

如果您需要将初始环境传递给孩子，您只需将其传递到 args 或 kwargs 列表中：

If you need to pass an initial environment to the child, you would just pass it in the args or kwargs list:

def myfunc(env=None):
    time.sleep(3)
    if env is not None:
        os.environ = env
    print os.environ['FOO']


if __name__ == "__main__":
    child_env = os.environ.copy()

    for i in range(3):
        child_env['FOO'] = "foo%s" % (i,)
        proc = multiprocessing.Process(target=myfunc, kwargs ={'env' : child_env})
        proc.start()

输出：

foo0
foo1
foo2

请注意，如果您使用 multiprocessing.Pool ，则可以使用初始化器 / initargs 要设置的关键字参数在池中的每个进程开始时一次正确的环境：

Note that if you're using a multiprocessing.Pool, you can use the initializer/initargs keyword arguments to just set the correct environment once at the start of each process in the pool:

def init(env):
    os.environ = env

def myfunc():
    print os.environ['FOO']


if __name__ == "__main__":
    child_env = os.environ.copy()
    child_env['FOO'] = "foo"
    pool = multiprocessing.Pool(initializer=init, initargs=(child_env,))
    pool.apply(myfunc,()) 

输出：

foo

这篇关于设置env var在Python中的多进程处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！