Python 使用多处理 [英] Python Using Multiprocessing

问题描述

我正在尝试在 python 3.6 中使用多处理.我有一个 for 循环，它运行一个带有不同参数的 method.目前，它一次运行一个，这需要花费相当多的时间，所以我正在尝试使用多处理.这是我所拥有的:

I am trying to use multiprocessing in python 3.6. I have a for loopthat runs a method with different arguments. Currently, it is running one at a time which is taking quite a bit of time so I am trying to use multiprocessing. Here is what I have:

def test(self):
    for key, value in dict.items():
        pool = Pool(processes=(cpu_count() - 1))
        pool.apply_async(self.thread_process, args=(key,value))
        pool.close()
        pool.join()


def thread_process(self, key, value):
    # self.__init__()
    print("For", key)

我认为我的代码使用 3 个进程来运行一个 method 但我想每个进程运行 1 个方法，但我不知道这是如何完成的.顺便说一句，我正在使用 4 核.

I think what my code is using 3 processes to run one method but I would like to run 1 method per process but I don't know how this is done. I am using 4 cores btw.

推荐答案

您在 for 循环的每次迭代中创建一个池.预先创建一个池，应用您想在多处理中运行的进程，然后加入它们:

You're making a pool at every iteration of the for loop. Make a pool beforehand, apply the processes you'd like to run in multiprocessing, and then join them:

from multiprocessing import Pool, cpu_count
import time

def t():
    # Make a dummy dictionary
    d = {k: k**2 for k in range(10)}

    pool = Pool(processes=(cpu_count() - 1))

    for key, value in d.items():
        pool.apply_async(thread_process, args=(key, value))

    pool.close()
    pool.join()


def thread_process(key, value):
    time.sleep(0.1)  # Simulate a process taking some time to complete
    print("For", key, value)

if __name__ == '__main__':
    t()

这篇关于Python 使用多处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！