使用并行线程提高Python执行速度 [英] Improving Python execution speed with parallel threads

问题描述

假设我有以下示例代码:

Let's say I have this sample code:

x = foo1(something1)
y = foo2(something2)

z = max(x, y)

我想通过使用线程来缩短此代码的执行时间(希望对您有所帮助吗?).我想使事情尽可能简单，所以基本上我想做的是创建两个同时工作的线程，分别计算foo1和foo2.

I want to improve the execution time of this code by using threads (hope it helps isn't it?). I'd like to keep things as simple as possible so basically what I'd like to do is creating two threads working at the same time which compute respectively foo1 and foo2.

我正在阅读一些有关线程的内容，但是我发现它有些棘手，并且我不会因为做这样简单的事情而在其中浪费太多时间.

I'm reading something about threads but I found it a little tricky and I can't lose too much time in it just for doing such a simple thing.

推荐答案

假定foo1或foo2受CPU限制，线程并不会缩短执行时间……实际上，通常会使情况变得更糟. ..有关更多信息，请参见 David Beazley关于全球口译员的PyCon2010演示文稿锁定/ Pycon2010 GIL幻灯片.该演示非常有用，我强烈建议所有试图在CPU内核之间分配负载的人使用它.

Assuming foo1 or foo2 is CPU-bound, threading doesn't improve the execution time... in fact, it normally makes it worse... for more information, see David Beazley's PyCon2010 presentation on the Global Interpreter Lock / Pycon2010 GIL slides. This presentation is very informative, I highly recommend it to anyone trying to distribute load across CPU cores.

提高性能的最好方法是使用多处理模块

The best way to improve performance is with the multiprocessing module

假设foo1()和foo2()之间不需要共享状态，这样做可以提高执行性能...

Assuming there is no shared state required between foo1() and foo2(), do this to improve execution performance...

from multiprocessing import Process, Queue
import time

def foo1(queue, arg1):
    # Measure execution time and return the total time in the queue
    print "Got arg1=%s" % arg1
    start = time.time()
    while (arg1 > 0):
        arg1 = arg1 - 1
        time.sleep(0.01)
    # return the output of the call through the Queue
    queue.put(time.time() - start)

def foo2(queue, arg1):
    foo1(queue, 2*arg1)

_start = time.time()
my_q1 = Queue()
my_q2 = Queue()

# The equivalent of x = foo1(50) in OP's code
p1 = Process(target=foo1, args=[my_q1, 50])
# The equivalent of y = foo2(50) in OP's code
p2 = Process(target=foo2, args=[my_q2, 50])

p1.start(); p2.start()
p1.join(); p2.join()
# Get return values from each Queue
x = my_q1.get()
y = my_q2.get()

print "RESULT", x, y
print "TOTAL EXECUTION TIME", (time.time() - _start)

在我的机器上，结果为:

From my machine, this results in:

mpenning@mpenning-T61:~$ python test.py 
Got arg1=100
Got arg1=50
RESULT 0.50578212738 1.01011300087
TOTAL EXECUTION TIME 1.02570295334
mpenning@mpenning-T61:~$ 

这篇关于使用并行线程提高Python执行速度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！