如何并行运行函数? [英] How to run functions in parallel?

问题描述

我先进行了研究，但找不到我的问题的答案.我正在尝试在 Python 中并行运行多个函数.

I researched first and couldn't find an answer to my question. I am trying to run multiple functions in parallel in Python.

我有这样的事情:

files.py

import common #common is a util class that handles all the IO stuff

dir1 = 'C:\folder1'
dir2 = 'C:\folder2'
filename = 'test.txt'
addFiles = [25, 5, 15, 35, 45, 25, 5, 15, 35, 45]

def func1():
   c = common.Common()
   for i in range(len(addFiles)):
       c.createFiles(addFiles[i], filename, dir1)
       c.getFiles(dir1)
       time.sleep(10)
       c.removeFiles(addFiles[i], dir1)
       c.getFiles(dir1)

def func2():
   c = common.Common()
   for i in range(len(addFiles)):
       c.createFiles(addFiles[i], filename, dir2)
       c.getFiles(dir2)
       time.sleep(10)
       c.removeFiles(addFiles[i], dir2)
       c.getFiles(dir2)

我想调用 func1 和 func2 并让它们同时运行.这些函数不会相互交互或在同一对象上交互.现在我必须在 func2 启动之前等待 func1 完成.我如何做类似下面的事情:

I want to call func1 and func2 and have them run at the same time. The functions do not interact with each other or on the same object. Right now I have to wait for func1 to finish before func2 to start. How do I do something like below:

process.py

from files import func1, func2

runBothFunc(func1(), func2())

我希望能够几乎同时创建两个目录，因为我每分钟都在计算正在创建的文件数量.如果目录不存在，它会耽误我的时间.

I want to be able to create both directories pretty close to the same time because every min I am counting how many files are being created. If the directory isn't there it will throw off my timing.

推荐答案

你可以使用 threading 或 multiprocessing.

You could use threading or multiprocessing.

由于CPython 的特殊性，线程 不太可能实现真正的并行性.因此，multiprocessing 通常是更好的选择.

Due to peculiarities of CPython, threading is unlikely to achieve true parallelism. For this reason, multiprocessing is generally a better bet.

这是一个完整的例子:

from multiprocessing import Process

def func1():
  print 'func1: starting'
  for i in xrange(10000000): pass
  print 'func1: finishing'

def func2():
  print 'func2: starting'
  for i in xrange(10000000): pass
  print 'func2: finishing'

if __name__ == '__main__':
  p1 = Process(target=func1)
  p1.start()
  p2 = Process(target=func2)
  p2.start()
  p1.join()
  p2.join()

启动/加入子进程的机制可以很容易地封装到一个函数中，就像你的runBothFunc:

The mechanics of starting/joining child processes can easily be encapsulated into a function along the lines of your runBothFunc:

def runInParallel(*fns):
  proc = []
  for fn in fns:
    p = Process(target=fn)
    p.start()
    proc.append(p)
  for p in proc:
    p.join()

runInParallel(func1, func2)

这篇关于如何并行运行函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！