Python Multiprocessing-将类方法应用于对象列表 [英] Python Multiprocessing - apply class method to a list of objects

问题描述

有没有一种简单的方法可以使用Multiprocessing来做到这一点?

Is there a simple way to use Multiprocessing to do the equivalent of this?

for sim in sim_list:
  sim.run()

其中sim_list的元素是模拟"对象，而run()是模拟类的一种方法，这样做会修改对象的属性.例如:

where the elements of sim_list are "simulation" objects and run() is a method of the simulation class which does modify the attributes of the objects. E.g.:

class simulation:
    def __init__(self):
        self.state['done']=False
        self.cmd="program"
    def run(self):
        subprocess.call(self.cmd)
        self.state['done']=True

sim_list中的所有sim都是独立的，因此该策略不必是线程安全的.

All the sim in sim_list are independent, so the strategy does not have to be thread safe.

我尝试了以下方法，这显然是有缺陷的，因为该参数是由Deepcopy传递的，而不是就地修改的.

I tried the following, which is obviously flawed because the argument is passed by deepcopy and is not modified in-place.

from multiprocessing import Process

for sim in sim_list:
  b = Process(target=simulation.run, args=[sim])
  b.start()
  b.join()

推荐答案

一种执行所需操作的方法是让您的计算类(在您的情况下为simulation)成为Process的子类.正确初始化后，此类的实例将在单独的进程中运行，您可以根据需要从列表中选择一组实例.

One way to do what you want is to have your computing class (simulation in your case) be a subclass of Process. When initialized properly, instances of this class will run in separate processes and you can set off a group of them from a list just like you wanted.

以下是一个示例，该示例基于您在上面的内容编写而成:

Here's an example, building on what you wrote above:

import multiprocessing
import os
import random

class simulation(multiprocessing.Process):
    def __init__(self, name):
        # must call this before anything else
        multiprocessing.Process.__init__(self)

        # then any other initialization
        self.name = name
        self.number = 0.0
        sys.stdout.write('[%s] created: %f\n' % (self.name, self.number))

    def run(self):
        sys.stdout.write('[%s] running ...  process id: %s\n' 
                         % (self.name, os.getpid()))

        self.number = random.uniform(0.0, 10.0)
        sys.stdout.write('[%s] completed: %f\n' % (self.name, self.number))

然后只列出对象并以循环开头:

Then just make a list of objects and start each one with a loop:

sim_list = []
sim_list.append(simulation('foo'))
sim_list.append(simulation('bar'))

for sim in sim_list:
    sim.start()

运行此命令时，您应该看到每个对象都在其自己的进程中运行.不要忘记在类初始化之前首先调用Process.__init__(self).

When you run this you should see each object run in its own process. Don't forget to call Process.__init__(self) as the very first thing in your class initialization, before anything else.

很显然，在此示例中，我没有包含任何进程间通信.您必须添加，如果您的情况需要它(不清楚您是否需要).

Obviously I've not included any interprocess communication in this example; you'll have to add that if your situation requires it (it wasn't clear from your question whether you needed it or not).

这种方法对我来说效果很好，我也没有发现任何缺点.如果有人知道我忽略的隐患，请告诉我.

This approach works well for me, and I'm not aware of any drawbacks. If anyone knows of hidden dangers which I've overlooked, please let me know.

我希望这会有所帮助.

这篇关于Python Multiprocessing-将类方法应用于对象列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！