在python中进行多处理时更改全局变量 [英] changing global variable when multiprocessing in python

问题描述

所以我最终想要做的是读取一行，对该行中的信息进行一些计算，然后将结果添加到某些全局对象中，但是我似乎永远无法使它起作用.例如，下面的代码中test始终为0.我知道这是错误的，并且我尝试了其他方法，但是仍然无法正常工作.

So what I am trying to do ultimately is read a line, do some calculations with the info in that line, then add the result to some global object, but I can never seem to get it to work. For instance, test is always 0 in the code below. I know this is wrong, and I have tried doing it other ways, but it still isn't working.

import multiprocessing as mp

File = 'HGDP_FinalReport_Forward.txt'
#short_file = open(File)
test = 0

def pro(temp_line):
    global test
    temp_line = temp_line.strip().split()
    test = test + 1
    return len(temp_line)

if __name__ == "__main__":
    with open("HGDP_FinalReport_Forward.txt") as lines:
        pool = mp.Pool(processes = 10)
        t = pool.map(pro,lines.readlines())

推荐答案

池产生的工作进程将获得自己的全局变量副本并进行更新.除非您明确设置，否则它们不会共享内存.最简单的解决方案是将test的最终值传达回主过程，例如通过返回值.像( unested )一样:

The worker processes spawned by the pool get their own copy of the global variable and update that. They don't share memory unless you set that up explicitly. The easiest solution is to communicate the final value of test back to the main process, e.g. via the return value. Something like (untested):

def pro(temp_line):
    test = 0
    temp_line = temp_line.strip().split()
    test = test + 1
    return test, len(temp_line)

if __name__ == "__main__":
    with open("somefile.txt") as lines:
        pool = mp.Pool(processes = 10)
        tests_and_t = pool.map(pro,lines.readlines())
        tests, t = zip(*test_and_t)
        test = sum(tests)

这篇关于在python中进行多处理时更改全局变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！